Context sensing hierarchical instruction learning method and device for large language model

Abstract

The invention discloses a context sensing hierarchical instruction learning method and device oriented to a large language model, and relates to the technical field of artificial intelligence. The method comprises the steps of distinguishing instruction roles of a large language model for initial training data, and constructing training data according to the instruction roles; constructing an instruction role mark for an instruction role in the training data, and determining instruction segment segmentation features and instruction segment query features according to the instruction role mark, instruction segment segmentation embedding and instruction segment query embedding; inputting the instruction segment segmentation feature, the instruction segment query feature and the Token embedding feature into a context sensing module to obtain sample input data; inputting the sample input data into the large language model, and performing supervision fine tuning training on the reply model to obtain a trained reply model; and inputting the to-be-replied text into the trained reply model to obtain dialogue reply data. By adopting the method and the device, the execution priority degree of the big language model on the key role instruction can be improved, and the security and the stability of the dialogue application are ensured.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a context-aware hierarchical instruction learning method and device for a large language model. Background Art

[0002] Large Language Models (LLMs) have demonstrated tremendous potential in recent years across many fields, particularly in applications such as intelligent agents, automated decision-making, educational tools, and medical assistance. LLMs are capable of handling complex language tasks, including generating natural language text, answering questions, and executing commands. However, existing LLM architectures treat all input tokens uniformly, lacking a mechanism to differentiate and prioritize commands based on their role or importance. This flaw makes the model vulnerable to malicious manipulation. Attackers can inject malicious commands to overwrite legitimate commands, posing security risks and causing the model to output irresponsible or dangerous responses.

[0003] To solve this problem, a method can be adopted to clearly distinguish system instructions, user prompts and data input, and follow the order according to priority, so that system instructions have higher priority than other instructions to ensure that the model can correctly and safely perform functions as intended by the system designer and provide reliable assistance to users.

[0004] In order to achieve the above functions and enhance the security of large language models, many studies have proposed different solutions. Existing technologies, such as the data-driven model fine-tuning method proposed by OpenAI, use a hierarchical instruction dataset to train the model, thereby ensuring that instructions between different roles follow the priority. However, this method fails to perform explicit role hierarchy distinction and priority management for all tokens, resulting in the model being unable to effectively deal with the interference of malicious instructions. The instruction segment embedding method proposed by the instruction segmentation embedding (ISE) technology sums the features of the instruction segment with the token embedding and uniformly performs semantic offset on each role segment. However, this method ignores the fine-grained semantic information at the token level, thereby limiting the model's accurate understanding of role-level instructions. Summary of the Invention

[0005] To address the technical issues in existing technologies, such as the lack of token-level semantic information when explicitly modeling instruction role features, which leads to inaccurate understanding of role-level instructions by the model, the present invention provides a context-aware hierarchical instruction learning method and device for large language models. The technical solution is as follows:

[0006] In one aspect, a context-aware hierarchical instruction learning method for a large language model is provided. The method is implemented by a context-aware hierarchical instruction learning device for a large language model. The method includes:

[0007] S1. Obtain initial training data, distinguish the instruction roles of the large language model from the initial training data, construct training data based on the instruction roles, and determine the instruction segment segmentation embedding, instruction segment query embedding, and token embedding of the input sequence in the training data;

[0008] S2. Construct instruction role tags for instruction roles in the training data, determine instruction segment segmentation features based on the instruction role tags and instruction segment segmentation embedding, determine instruction segment query features based on the instruction role tags and instruction segment query embedding, and determine token embedding features based on token embedding;

[0009] S3. Input the instruction segment segmentation features, instruction segment query features, and token embedding features into the context perception module to obtain sample input data.

[0010] S4. Input the sample input data into the large language model, and perform supervised fine-tuning training on the response model composed of the context-aware module and the large language model to obtain a trained response model;

[0011] S5. Obtain the text to be replied, input the text to be replied into the trained reply model, and obtain the dialogue reply data of the text to be replied.

[0012] On the other hand, a context-aware hierarchical instruction learning device for a large language model is provided. The device is applied to a context-aware hierarchical instruction learning method for a large language model. The device includes:

[0013] A construction unit is used to obtain initial training data, distinguish the instruction roles of the large language model from the initial training data, construct training data based on the instruction roles, and determine the instruction segment segmentation embedding, instruction segment query embedding, and token embedding features of the input sequence in the training data;

[0014] a determination unit, configured to construct an instruction role tag for the instruction role in the training data, determine the instruction segment segmentation feature based on the instruction role tag and the instruction segment segmentation embedding, and determine the instruction segment query feature based on the instruction role tag and the instruction segment query embedding;

[0015] The context perception unit is used to input the instruction segment segmentation features, instruction segment query features, and token embedding features into the context perception module to obtain sample input data;

[0016] A fine-tuning training unit is used to input sample input data into the large language model, perform supervised fine-tuning training on the response model composed of the context-aware module and the large language model, and obtain a trained response model;

[0017] The reply unit is used to obtain the text to be replied, input the text to be replied into the trained reply model, and obtain the dialogue reply data of the text to be replied.

[0018] On the other hand, a context-aware hierarchical instruction learning device for a large language model is provided, and the context-aware hierarchical instruction learning device for a large language model includes: a processor; a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned context-aware hierarchical instruction learning methods for a large language model is implemented.

[0019] On the other hand, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, and the at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned context-aware hierarchical instruction learning methods for large language models.

[0020] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0021] The embodiment of the present invention realizes context-aware instruction hierarchy compliance priority, effectively distinguishes different instruction roles through information interaction between instruction segments of different roles and the entire context, and correctly learns the instruction compliance hierarchy based on the dialogue data training model, thereby improving the execution priority of the large language model for key role instructions, preventing the model from generating unsafe responses when vulnerable low-priority instructions are attacked, improving the security of the model, and maintaining the instruction compliance capability of the large language model, that is, maintaining the effectiveness of the model. At the same time, distinguishing different instruction roles also provides flexible expansion for the large language model as the basis for dialogue systems and intelligent agent applications, ensuring the security and stability of such dialogue applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1-1 This is a flow chart of a context-aware hierarchical instruction learning method for a large language model provided by an embodiment of the present invention;

[0024] Figure 1-2This is a computational diagram of a context-aware hierarchical instruction learning method for a large language model provided by an embodiment of the present invention;

[0025] Figure 2 This is a block diagram of a context-aware hierarchical instruction learning device for a large language model provided by an embodiment of the present invention;

[0026] Figure 3 This is a structural diagram of a context-aware hierarchical instruction learning device for a large language model provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0028] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as an "exemplary" in the present invention should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner. Furthermore, in the embodiments of the present invention, "and / or" can mean both or either of the two.

[0029] In the embodiments of the present invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the meanings they convey are the same. The terms "of," "corresponding," and "corresponding" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the meanings they convey are the same.

[0030] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0031] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0032] The embodiment of the present invention provides a context-aware hierarchical instruction learning method for a large language model. The method can be implemented by a context-aware hierarchical instruction learning device for a large language model. The context-aware hierarchical instruction learning device for a large language model can be a terminal or a server. Figure 1-1 The flowchart of the context-aware hierarchical instruction learning method for large language models is shown as follows: Figure 1-2 The computational diagram of the context-aware hierarchical instruction learning method for a large language model is shown. The processing flow of the method may include the following steps:

[0033] S1. Obtain initial training data, distinguish the instruction roles of the large language model for the initial training data, construct training data based on the instruction roles, and determine the instruction segment segmentation embedding, instruction segment query embedding, and token embedding of the input sequence in the training data.

[0034] Among them, according to the existing large language model method, determining the token embedding (i.e. word embedding) means constructing the token embedding matrix ,in is the vocabulary length, Determining the instruction segment segmentation embedding and instruction segment query embedding of the input sequence in the training data means constructing two other matrices according to the Token embedding matrix construction method: the instruction segment segmentation embedding matrix , instruction segment query embedding matrix , is the number of role categories.

[0035] Among them, the instruction roles of the large language model are pre-set with different priorities, and different priorities indicate different instruction execution orders.

[0036] In one feasible implementation, the roles of large language model instructions are distinguished, which means that during the data cleaning and annotation stage, the different roles in the conversation between the large language model and the user are manually classified, such as system, user, tool and model response, and their role in the conversation is determined according to the different roles.

[0037] Specifically, typical dialogue roles include: system, which is responsible for guiding the dialogue and providing basic functions, and usually has the highest instruction priority, such as system instructions or constraints that define model behavior; user, which interacts by inputting questions or requests. User instructions usually follow the rules set by the system and have lower priority than system instructions in most scenarios; tool, which refers to auxiliary tools used in the dialogue, such as external knowledge base, database, computing module, etc. Tool instructions are usually used to assist the model in performing tasks, but have lower priority than system and user instructions; model / agent / assistant: as an entity that executes instructions, it generates responses based on the received instructions or performs specific tasks to generate reply responses. For non-inference models, it is reply content, and for inference models, it includes inference content and reply content. These roles usually handle the interaction between the system and the user. The reply results participate in the subsequent dialogue as dialogue history and continue to perform tasks based on the preset rules and algorithms of the model itself.

[0038] The command roles of the large language model are pre-set with different priorities. This means that the command roles need to be divided into a hierarchy of command compliance. That is, a hierarchy of command compliance is established based on the different command roles. Commands from high-priority roles are executed first (such as system commands), while commands from low-priority roles are executed according to the constraints of the higher priority (such as tool commands). In this structure, system commands have the highest priority and must always be executed first to ensure that the model follows core rules when executing tasks. User commands typically have a lower priority than system commands, but still need to be followed during execution. Tool commands are executed according to actual task requirements and are usually subject to the constraints of system and user commands. Model commands are the model's response history, serving as a reference for subsequent conversations. During implementation, the hierarchical division of command roles based on the above rules ensures that various commands are processed according to their priority, preventing malicious or inappropriate commands from interfering with certain commands.

[0039] Optionally, the constructed training data includes aligned dialogue data and non-aligned dialogue data, where:

[0040] Aligned dialogue data represents dialogues between instruction roles that follow preset priority rules; unaligned dialogue data represents dialogues between instruction roles that do not follow priority rules or have priority conflicts.

[0041] In one feasible implementation, aligned and unaligned conversation data constructed based on instruction roles is constructed by leveraging role information in the training dataset to construct conversation data between different roles. Aligned conversation data refers to conversations between roles that follow preset rules and priorities. For example, the system provides task instructions to the model, the user asks questions based on the system's framework, and the tool provides auxiliary information. All role instructions are executed according to a prioritized hierarchy. Unaligned conversation data refers to conversations between roles that do not strictly follow these rules or that may contain instruction conflicts. For example, instruction injection attacks or jailbreak attacks can cause low-priority instructions to bypass system instructions, thereby impacting the normal execution of the model. By constructing unaligned data, the model learns how to handle instruction priority conflicts, ensuring the correct execution of various instructions in a multi-role dialogue environment. During subsequent fine-tuning training, responses from aligned conversation data should correctly follow instruction priorities and execute tasks assigned by all roles. Responses from unaligned conversation data should identify conflicts between instructions and only execute tasks assigned by the highest-priority role. These two sets of conversation data are used to train the model to correctly handle instruction priorities.

[0042] S2. Construct instruction role tags for the instruction roles in the training data, determine instruction segment segmentation features based on the instruction role tags and instruction segment segmentation embedding, and determine instruction segment query features based on the instruction role tags and instruction segment query embedding.

[0043] Optionally, construct instruction role labels for instruction roles in the training data, including:

[0044] S21. Mark the role information corresponding to each Token in the input sequence of the training data to form an instruction role label sequence.

[0045] In a feasible implementation, different instruction tags are assigned to each instruction role (such as system, user, data, output, etc.), and the role information corresponding to each Token is marked in the input sequence, that is, for the input Token sequence , forming the instruction role tag sequence as follows (1):

[0046] (1)

[0047] in , is the number of role categories.

[0048] Optionally, determining instruction segment segmentation features based on the instruction role tag and the instruction segment segmentation embedding includes:

[0049] S22, segmentation and embedding according to instruction segment , the role instruction sequence Through table lookup operation, Map to The i-th dimension vector of is used to map the instruction role label to a feature vector and obtain the instruction segment segmentation feature.

[0050] Based on the instruction role tag and instruction segment query embedding, the instruction segment query features are determined, including:

[0051] S23, query embedding according to instruction segment , the role instruction sequence Through table lookup operation, Map to The i-th dimension vector of is used to map the instruction role label to a feature vector and obtain the instruction segment query feature.

[0052] In one feasible implementation, the learnable instruction segments can be segmented and embedded into Indicates that the instruction segment query is embedded in the Indicates that the instruction role tag sequence Mapped into feature vectors, the instruction segment segmentation features of the input sequence are used Indicates that the instruction segment query feature is used Represented by, where N is the sequence length, d is the model vector dimension, and Token embedding is Indicates that is the length of the vocabulary. Based on the existing large language model Token processing technology, the Token sequence x is mapped to the Token embedding feature .

[0053] S3. Input the instruction segment segmentation features, instruction segment query features, and token embedding features into the context perception module to obtain sample input data.

[0054] Optionally, the context awareness module includes an intra-segment summarization module and a context aggregation module.

[0055] S3 inputs the instruction segmentation features, instruction segment query features, and token embedding features into the context perception module to obtain sample input data, including:

[0056] S31. Input the instruction segment query feature and the token embedding feature into the intra-segment summary module, and fuse the instruction segment query feature and the token embedding feature through the cross-attention mechanism of the intra-segment summary module to obtain the fused feature.

[0057] In a feasible implementation, the two features of the instruction segment query feature and the token embedding feature are first projected into the attention space as shown in the following formula (2):

[0058] (2)

[0059] in are different, learnable weight parameters, Indicates the Token embedding feature; represents query embedding, Indicates the key, Represents a value;

[0060] Then the two features are fused through cross attention, as shown in the following formula (3):

[0061] (3)

[0062] Where d represents the dimension of query embedding, and M is the intra-segment attention mask matrix. The intra-segment attention mask matrix M is expressed as follows: the mask matrix corresponding to the same segment Token is 0, and the mask matrix corresponding to different segment Token is , forming a block diagonal matrix. The calculation result of the intra-segment summary module is the fusion feature, which is expressed by the following formula (4):

[0063] (4)

[0064] in Contains the semantic and hierarchical information within the segment.

[0065] S32. Input the fused features into the context aggregation module, aggregate all context information through the self-attention mechanism, and obtain the context-level learning output.

[0066] In one feasible implementation, the fusion feature Projected into the attention space, as shown in Equation (5):

[0067] (5)

[0068] in, 、 and are different, learnable weight parameters.

[0069] Aggregating all contextual information through the self-attention mechanism is shown in the following formula (6):

[0070] (6)

[0071] in, It represents the context-level learning output, integrates the global interaction of the overall token between instruction segments, and contains the semantic and hierarchical information of the full text.

[0072] S33. Integrate the instruction segment segmentation features, token embedding features, and context-level learning output to obtain sample input data.

[0073] Optionally, S33 integrates the instruction segment segmentation features, the token embedding features, and the context-level learning output to obtain sample input data, including:

[0074] The instruction segmentation features, token embedding features, and the product of the context-level learning output and the balance parameter are added to obtain the sample input data.

[0075] In a feasible implementation, the integration method is as follows (7):

[0076] (7)

[0077] in is a learnable balance parameter.

[0078] The embodiment of the present invention can improve the semantic understanding and context perception capabilities of role instructions of a large language model by constructing a context-aware hierarchical instruction learning method for a large language model.

[0079] S4. Input the sample input data into the large language model, perform supervised fine-tuning training on the response model composed of the context-aware module and the large language model, and obtain a trained response model.

[0080] Among them, typical large language models may include LLaMa, DeepSeek, Qwen, etc. Large language models refer to large-scale pre-trained language models based on deep neural networks, especially Transformer architecture, which can be trained with large amounts of text data and achieve outstanding performance in natural language processing tasks. Large language models have gained a deep understanding of language through learning from massive corpora, and are able to generate coherent text, perform contextual reasoning, answer questions, translate languages, etc. In the present invention, large language models are used to process complex language tasks, adapt to various application scenarios through fine-tuning or a small amount of contextual information, and generate smooth, accurate, and secure dialogue responses.

[0081] In one feasible implementation, the overall model is trained through a supervised fine-tuning method using instruction-role dialogue data, and the model learns the priority of hierarchical instruction following through aligned and unaligned data.

[0082] Supervised fine-tuning methods train models using data with clear priority labels. Specifically, supervised fine-tuning methods use conversation data with standard responses to optimize the model's behavior, enabling it to learn to follow instructions from different roles based on the priority of the instructions and the correct responses. Typical training methods include instruction fine-tuning and preference optimization. Instruction fine-tuning helps the model learn how to handle instructions based on priority between different roles by providing clear examples of instructions and responses. Preference optimization introduces positive and negative samples and constructs preference pairs using instruction priority labels. For aligned conversation data, positive samples refer to models that correctly respond to aligned conversations and have better response quality, while negative samples refer to models that incorrectly understand instruction priority or have poor response quality. For unaligned conversation data, positive samples refer to models that correctly identify instruction priority conflicts in conversations and reject unaligned conversations, while negative samples refer to models that fail to identify instruction priority conflicts or fail to reject responses to unaligned conversations.

[0083] Compared with existing mainstream methods, the embodiments of the present invention can explicitly model and perform neural network calculations on instruction role features, and can clearly learn instruction role features based on instruction semantics, and then input them into a large language model to utilize the capabilities of the basic model, making the instruction role hierarchy clearer, so as to achieve better role instruction compliance priority and provide better security and application performance for large language model dialogue applications.

[0084] S5. Obtain the text to be replied, input the text to be replied into the trained reply model, and obtain the conversation reply data of the text to be replied.

[0085] In a feasible implementation, after obtaining the text to be replied, the instruction role of the text to be replied is determined, and the instruction segment segmentation embedding, instruction segment query embedding, and token embedding of the text to be replied are determined. Based on the instruction role of the text to be replied, the instruction segment segmentation embedding, and instruction segment query embedding of the text to be replied, the instruction segment segmentation features and instruction segment query features of the text to be replied, as well as the token embedding, are obtained respectively. The specific processing flow can refer to the above step S2 and will not be repeated here.

[0086] The command segmentation features, command query features, and token embedding features of the reply text are fed into the trained context-aware module to obtain the context-level learning output of the reply text. The processing flow of this module can be referred to above in step S3 and is not detailed here. The command segmentation features, token embedding features, and context-level learning output are integrated and fed into the trained large language model to obtain the conversation response data for the reply text.

[0087] The embodiment of the present invention realizes context-aware instruction hierarchy compliance priority, effectively distinguishes different instruction roles through information interaction between instruction segments of different roles and the entire context, and correctly learns the instruction compliance hierarchy based on the dialogue data training model, thereby improving the execution priority of the large language model for key role instructions, preventing the model from generating unsafe responses when vulnerable low-priority instructions are attacked, improving the security of the model, and maintaining the instruction compliance capability of the large language model, that is, maintaining the effectiveness of the model. At the same time, distinguishing different instruction roles also provides flexible expansion for the large language model as the basis for dialogue systems and intelligent agent applications, ensuring the security and stability of such dialogue applications.

[0088] Figure 2 This is a block diagram of a context-aware hierarchical instruction learning device for a large language model provided by an embodiment of the present invention. The device is used for a context-aware hierarchical instruction learning method for a large language model. Figure 2 The apparatus includes a construction unit 210, a determination unit 220, a context perception unit 230, a fine-tuning training unit 240, and a response unit 250.

[0089] A construction unit 210 is configured to obtain initial training data, distinguish instruction roles of the large language model from the initial training data, construct training data based on the instruction roles, and determine instruction segment segmentation embedding, instruction segment query embedding, and token embedding for input sequences in the training data;

[0090] A determination unit 220 is configured to construct instruction role labels for instruction roles in the training data, determine instruction segment segmentation features based on the instruction role labels and instruction segment segmentation embeddings, determine instruction segment query features based on the instruction role labels and instruction segment query embeddings, and determine token embedding features based on the token embeddings;

[0091] The context perception unit 230 is used to input the instruction segmentation feature, the instruction segment query feature, and the token embedding feature into the context perception module to obtain sample input data;

[0092] A fine-tuning training unit 240 is configured to input the sample input data into the large language model, and perform supervised fine-tuning training on the response model composed of the context-aware module and the large language model to obtain a trained response model;

[0093] The reply unit 250 is used to obtain the text to be replied, input the text to be replied into the trained reply model, and obtain the dialogue reply data of the text to be replied.

[0094] Figure 3 This is a structural diagram of a context-aware hierarchical instruction learning device for a large language model provided by an embodiment of the present invention. Figure 3 As shown, the context-aware hierarchical instruction learning device for a large language model may include the above Figure 2 The context-aware hierarchical instruction learning device for a large language model is shown. Optionally, the context-aware hierarchical instruction learning device 310 for a large language model may include a first processor 2001.

[0095] Optionally, the context-aware hierarchical instruction learning device 310 for a large language model may further include a memory 2002 and a transceiver 2003 .

[0096] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0097] The following combination Figure 3 The components of the context-aware hierarchical instruction learning device 310 for a large language model are described in detail:

[0098] The first processor 2001 is the control center of the context-aware hierarchical instruction learning device 310 for a large language model. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs) or one or more field programmable gate arrays (FPGAs).

[0099] Optionally, the first processor 2001 can perform various functions of the context-aware hierarchical instruction learning device 310 for a large language model by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002.

[0100] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 3 CPU0 and CPU1 are shown in FIG.

[0101] In a specific implementation, as an embodiment, the context-aware hierarchical instruction learning device 310 for a large language model may also include multiple processors, such as Figure 3 1 and 2. The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). A processor herein can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0102] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled by the first processor 2001 for execution. The specific implementation method can refer to the above method embodiment and will not be repeated here.

[0103] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001 or exist independently and access the memory 2002 through the interface circuit ( Figure 3 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0104] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0105] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 3 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0106] Optionally, the transceiver 2003 may be integrated with the first processor 2001 or may exist independently and be connected to the first processor 2001 through the interface circuit of the context-aware hierarchical instruction learning device 310 for a large language model ( Figure 3 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0107] It should be noted that Figure 3 The structure of the context-aware hierarchical instruction learning device 310 for a large language model shown in the figure does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0108] In addition, the technical effects of the context-aware hierarchical instruction learning device 310 for large language models can refer to the technical effects of the context-aware hierarchical instruction learning method for large language models described in the above method embodiment, and will not be repeated here.

[0109] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc.

[0110] It should also be understood that the memory in the embodiments of the present invention may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0111] The above embodiments can be implemented in whole or in part via software, hardware (e.g., circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions or computer programs. When loaded or executed on a computer, the processes or functions described in accordance with the embodiments of the present invention are fully or partially performed. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired means (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server or data center that contains a collection of one or more available media. The available medium can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media. The semiconductor media can be a solid-state drive.

[0112] It should be understood that the term "and / or" as used herein simply describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. A and B can be singular or plural. Furthermore, the character " / " as used herein generally indicates an "or" relationship between the associated objects, but it may also indicate an "and / or" relationship. For specific understanding, please refer to the context.

[0113] In this disclosure, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, "at least one of a, b, or c" can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0114] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0115] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0116] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0117] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of the device or unit, which can be electrical, mechanical or other forms.

[0118] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0119] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0120] If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or the portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical disks.

[0121] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A context-aware hierarchical instruction learning method for large language models, characterized by: The method comprises: S1. Obtain initial training data, distinguish the instruction roles of the large language model from the initial training data, construct training data based on the instruction roles, and determine the instruction segment segmentation embedding, instruction segment query embedding, and token embedding of the input sequence in the training data; S2. Construct instruction role tags for instruction roles in the training data, determine instruction segment segmentation features based on the instruction role tags and instruction segment segmentation embedding, determine instruction segment query features based on the instruction role tags and instruction segment query embedding, and determine token embedding features based on token embedding; S3. Input the instruction segment segmentation features, instruction segment query features, and token embedding features into the context perception module to obtain sample input data. S4. Input the sample input data into the large language model, and perform supervised fine-tuning training on the response model composed of the context-aware module and the large language model to obtain a trained response model; S5. Obtain the text to be replied, input the text to be replied into the trained reply model, and obtain the dialogue reply data of the text to be replied.

2. The context-aware hierarchical instruction learning method for a large language model according to claim 1, characterized in that The instruction roles of the large language model are pre-set with different priorities, and different priorities represent different instruction execution orders.

3. The context-aware hierarchical instruction learning method for a large language model according to claim 1, characterized in that The constructed training data includes aligned dialogue data and non-aligned dialogue data; The aligned dialogue data represents a dialogue between instruction characters following a preset priority rule; The non-aligned dialogue data represents a dialogue between instruction roles that does not follow priority rules or has priority conflicts.

4. The context-aware hierarchical instruction learning method for a large language model according to claim 1, characterized in that The step of constructing instruction role labels for instruction roles in the training data includes: The role information corresponding to each Token is marked in the input sequence of the training data to form an instruction role label sequence.

5. The context-aware hierarchical instruction learning method for a large language model according to claim 4, characterized in that: Determining the instruction segmentation features based on the instruction role label and the instruction segment segmentation embedding includes: Map the instruction role label to a feature vector based on the instruction segment segmentation embedding to obtain the instruction segment segmentation feature; Determining the instruction segment query feature based on the instruction role tag and the instruction segment query embedding includes: According to the instruction segment query embedding, the instruction role label is mapped into a feature vector to obtain the instruction segment query feature.

6. The context-aware hierarchical instruction learning method for a large language model according to claim 1, characterized in that The context awareness module includes an intra-segment summary module and a context aggregation module; The step S3 inputs the instruction segment segmentation features, instruction segment query features, and token embedding features into the context perception module to obtain sample input data, including: S31, input the instruction segment query feature and the token embedding feature into the intra-segment summary module, and fuse the instruction segment query feature and the token embedding feature through the cross attention mechanism of the intra-segment summary module to obtain a fused feature; S32, input the fused features into the context aggregation module, aggregate all context information through the self-attention mechanism, and obtain the context-level learning output; S33. Integrate the instruction segment segmentation features, token embedding features, and context-level learning output to obtain sample input data.

7. The context-aware hierarchical instruction learning method for a large language model according to claim 6, characterized in that: The step S33 integrates the instruction segmentation features, the token embedding features, and the context-level learning output to obtain sample input data, including: The instruction segmentation features, token embedding features, and the product of the context-level learning output and the balance parameter are added to obtain the sample input data.

8. A context-aware hierarchical instruction learning device for a large language model, the context-aware hierarchical instruction learning device for a large language model being used to implement the context-aware hierarchical instruction learning method for a large language model as claimed in any one of claims 1 to 7, characterized in that: The device comprises: A construction unit is used to obtain initial training data, distinguish the instruction roles of the large language model from the initial training data, construct training data based on the instruction roles, and determine the instruction segment segmentation embedding, instruction segment query embedding, and token embedding of the input sequence in the training data; A determination unit is used to construct an instruction role tag for the instruction role in the training data, determine the instruction segment segmentation feature based on the instruction role tag and the instruction segment segmentation embedding, determine the instruction segment query feature based on the instruction role tag and the instruction segment query embedding, and determine the token embedding feature based on the token embedding; The context perception unit is used to input the instruction segment segmentation features, instruction segment query features, and token embedding features into the context perception module to obtain sample input data; A fine-tuning training unit is used to input sample input data into the large language model, perform supervised fine-tuning training on the response model composed of the context-aware module and the large language model, and obtain a trained response model; The reply unit is used to obtain the text to be replied, input the text to be replied into the trained reply model, and obtain the dialogue reply data of the text to be replied.

9. A context-aware hierarchical instruction learning device for a large language model, characterized in that: The context-aware hierarchical instruction learning device for a large language model includes: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, which can be called by a processor to execute the method according to any one of claims 1 to 7.

Images