A slot value extraction method and device for slot meaning based on a large model

Abstract

Embodiments of the present application relate to a slot value extraction method and device based on slot meaning of a large model, related information of an mth slot at a current time is obtained; the related information is spliced from a to-be-filled sentence, an original input text and a fill-in-the-blank instruction; the to-be-filled sentence is composed of a slot name, a meaning explanation and a corresponding slot value of a first slot, a slot name, a meaning explanation and a corresponding slot value of a second slot, and a slot name, a meaning explanation and a corresponding slot value of a third slot; based on the related information of the mth slot at the current time, a parameter matrix, and a hidden layer representation vector of a decoder at a previous time, an attention matrix of the mth slot is determined; the parameter matrix includes q matrix, k matrix and v matrix in the attention mechanism; based on the attention matrix of the mth slot, a word table probability distribution corresponding to the mth slot is determined; based on the word table probability distribution, a slot value corresponding to the mth slot is determined, and a corresponding blank symbol is replaced to fill in a completed sentence.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method and apparatus for extracting slot values ​​based on the meaning of slots in a large model. Background Technology

[0002] Slot extraction is a crucial task in human-computer dialogue systems. It involves performing lexical and syntactic analysis on a given sentence within a specific domain, and then, in conjunction with the context, determining whether a given segment of the sentence string is a meaningful slot that fits the context. In short, the goal of slot extraction is to identify slot values ​​within user statements. In recent years, with the rapid development of deep learning technology, slot extraction has made significant progress, and researchers have proposed a series of effective algorithms. For example, current methods for slot extraction typically use text and the extraction command as input to the model, which outputs the slot name and corresponding slot value in a specific structure. However, these methods are prone to the following problems:

[0003] 1. Generates abnormal slots;

[0004] 2. The extracted slot values ​​are inaccurate.

[0005] Therefore, how to avoid generating abnormal slots or inaccurate slot values ​​during slot extraction is a technical problem that urgently needs to be solved. Summary of the Invention

[0006] Based on the above problems, this invention proposes a method and apparatus for extracting slot values ​​based on the meaning of slots in a large model, in order to solve the problems existing in the prior art.

[0007] To this end, in a first aspect, the present invention provides a method for extracting slot values ​​based on the meaning of slots in a large model, the method comprising:

[0008] Obtain relevant information for the m-th slot at the current time, wherein the relevant information is composed of the sentence to be filled, the original input text, and the fill-in instruction; the sentence to be filled consists of the slot name, meaning explanation, and corresponding slot value of the first slot, the slot name, meaning explanation, and corresponding slot value of the second slot, and the slot name, meaning explanation, and corresponding slot value of the third slot; the first slot is all slots before the current time, the second slot is the m-th slot at the current time, and the third slot is all slots after the current time, wherein the slot value corresponding to the second slot and the slot value corresponding to the third slot are replaced with fill-in symbols;

[0009] Based on the relevant information and parameter matrix of the m-th slot at the current time, and the hidden layer representation vector of the decoder at the previous time, the attention matrix of the m-th slot is determined; wherein, the parameter matrix includes the q matrix, k matrix and v matrix in the attention mechanism.

[0010] Based on the attention matrix of the m-th slot, determine the vocabulary probability distribution corresponding to the m-th slot;

[0011] Based on the probability distribution of the vocabulary, the slot value corresponding to the m-th slot is determined, and the corresponding fill-in symbol is replaced to fill in the completed sentence.

[0012] In one possible implementation, the relevant information is composed of a sentence, the original input text, and fill-in-the-blank instructions, specifically:

[0013] Input text: O\nSentence to be filled: U\nI

[0014] In the formula, I represents the fill-in-the-blank instruction, n represents the total number of slots, O represents the original input text, and U represents the sentence, expressed as:

[0015] The meaning of "s0" is Its corresponding value is _; ...; s m-1 The meaning is Its corresponding value is _; ...; s n-1 The meaning is Its corresponding value is _;

[0016] Where _ represents the position corresponding to the slot value; s n-1 This represents the name of the nth slot, and its corresponding interpretation is as follows: t n-1 This indicates the length of the text in the nth slot.

[0017] In one possible implementation, determining the vocabulary probability distribution corresponding to the m-th slot based on the attention matrix of the m-th slot specifically includes:

[0018] Based on the attention matrix of the m-th slot, determine the hidden layer representation vector output at the current time.

[0019] Based on the hidden layer representation vector output at the current time, determine the vocabulary probability distribution corresponding to the m-th slot.

[0020] In one possible implementation, the attention matrix of the m-th slot is determined based on the relevant information and parameter matrix of the m-th slot at the current time, and the hidden layer representation vector of the decoder at the previous time. Specifically, the attention matrix of the m-th slot is determined by the following formula:

[0021]

[0022] Among them, Atten m This represents the attention matrix corresponding to the m-th slot, o m-1W is the hidden layer representation vector of the decoder at the previous time step. v W q W k They are matrices v, q, and k, respectively, and d k Indicate i m and o m-1 The dimension, i m The encoding vector matrix represents the information of the m-th slot at the current time, and T represents the transpose.

[0023] In one possible implementation, determining the vocabulary probability distribution corresponding to the m-th slot based on the attention matrix of the m-th slot specifically involves determining the vocabulary probability distribution corresponding to the m-th slot according to the following formula:

[0024] prob vocb =softmax(o m )

[0025]

[0026] Among them, prob vovb Atten represents the probability distribution of the vocabulary. m This represents the attention matrix corresponding to the m-th slot. This represents the parameter matrix, used for linear transformations that convert the length of the attention output matrix to the vocabulary length. m This represents the hidden layer representation vector of the decoder at the current time step.

[0027] Secondly, the present invention provides a slot value extraction device based on the slot meaning of a large model, the device comprising:

[0028] The relevant information module is used to obtain the relevant information of the m-th slot at the current time. The relevant information is composed of the sentence to be filled, the original input text, and fill-in instructions. The sentence to be filled consists of the slot name, meaning explanation, and corresponding slot value of the first slot, the slot name, meaning explanation, and corresponding slot value of the second slot, and the slot name, meaning explanation, and corresponding slot value of the third slot. The first slot refers to all slots before the current time, the second slot is the m-th slot at the current time, and the third slot refers to all slots after the current time. The slot values ​​corresponding to the second and third slots are replaced with fill-in symbols.

[0029] The attention matrix module is used to determine the attention matrix of the m-th slot based on the relevant information and parameter matrix of the m-th slot at the current time, and the hidden layer representation vector of the decoder at the previous time; wherein, the parameter matrix includes the q matrix, k matrix and v matrix in the attention mechanism;

[0030] The vocabulary probability distribution module is used to determine the vocabulary probability distribution corresponding to the m-th slot based on the attention matrix of the m-th slot.

[0031] The slot value module is used to determine the slot value corresponding to the m-th slot based on the probability distribution of the vocabulary, and replace the corresponding fill symbol to fill in the completed sentence.

[0032] In one possible implementation, the vocabulary probability distribution module includes:

[0033] The first determining unit is used to determine the hidden layer representation vector output at the current time based on the attention matrix of the m-th slot.

[0034] The second determining unit is used to determine the vocabulary probability distribution corresponding to the m-th slot based on the hidden layer representation vector output at the current time.

[0035] Thirdly, the present invention provides a computer server, comprising: a memory, a processor, and a transceiver;

[0036] The processor is used to couple with the memory, read and execute instructions in the memory to implement the slot value extraction method based on the slot meaning of a large model as described in the first aspect.

[0037] The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

[0038] Fourthly, the present invention provides a chip system including a processor coupled to a memory, the memory storing program instructions, wherein when the program instructions stored in the memory are executed by the processor, the slot value extraction method based on the slot meaning of a large model as described in the first aspect is implemented.

[0039] Fifthly, the present invention provides a computer-readable storage medium storing a computer program, the computer program being executed by a processor of the slot value extraction method based on the slot meaning of a large model as described in the first aspect.

[0040] The slot value extraction method based on a large model provided by this invention organizes the slot name and slot value into a sentence during the slot information extraction process. The sentence, which contains an explanation of the meaning of each slot, is concatenated with the original input text and input into the large model. This allows the large model to fully understand the meaning of each slot, thereby accurately extracting the slot value corresponding to the slot and preventing the large model from outputting abnormal slots. Attached Figure Description

[0041] Figure 1This is a schematic diagram of the slot value extraction method based on a large model, provided in Embodiment 1 of the present invention.

[0042] Figure 2 A flowchart illustrating the process of determining the probability distribution of a vocabulary;

[0043] Figure 3 This is a schematic diagram of a slot value extraction device based on a large model for slot meaning provided in Embodiment 2 of the present invention;

[0044] Figure 4 This is a schematic diagram of the probability distribution module in this table;

[0045] Figure 5 This is a schematic diagram of the computer server structure provided in Embodiment 3 of the present invention;

[0046] Figure 6 This is a schematic diagram of the chip system structure provided in Embodiment 4 of the present invention;

[0047] Figure 7 This is a schematic diagram of the chip system structure provided in Embodiment 5 of the present invention. Detailed Implementation

[0048] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0049] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0050] Example 1

[0051] Embodiment 1 of this invention provides a slot extraction method based on a large model with structural enhancement. The execution entity of this application is a server, or a system or device with computational processing capabilities. For slot information extraction, based on a pre-trained large model, the slot name and slot value are organized into a sentence U using natural language text. This sentence U includes an explanation of the corresponding slot, and the slot value positions are marked with underscores "_". When the large model performs slot extraction, this sentence, the original input text O, and the fill-in instruction I are simultaneously input to the model. During the large model's decoding process, the large model can fully understand the meaning of each slot and accurately extract the corresponding slot value. The large model then outputs the sentence and fills the slot value marking positions with the extracted slot values. Here, "large model" refers to a large-scale language model, such as the common OpenAI ChatGPT, which will not be elaborated upon here. The focus is on a detailed description of the execution process of this application. Specifically, as follows... Figure 1 As shown, the method includes the following steps:

[0052] Step 110: Obtain the relevant information of the m-th slot at the current time;

[0053] The relevant information is composed of the sentence to be filled, the original input text, and the fill-in instruction. The sentence to be filled consists of the slot name, meaning explanation, and corresponding slot value of the first slot, the slot name, meaning explanation, and corresponding slot value of the second slot, and the slot name, meaning explanation, and corresponding slot value of the third slot. The first slot is all slots before the current time, the second slot is the m-th slot at the current time, and the third slot is all slots after the current time. The slot values ​​corresponding to the second slot and the third slot are replaced with fill-in symbols.

[0054] The following is a detailed explanation of splicing:

[0055] First, the predefined set of slot names S = {s0, s1, ..., s...} n-1}, where n represents the total number of slots, s 0≤l≤n-1 This represents the slot name of the (l+1)th slot. Additionally, we define V = {v0, v1, ..., v...} n-1} represents the set of slot values, v 0≤l≤n-1 s l The corresponding slot value.

[0056] Organize the slot name and slot value into a sentence U using natural language, where the slot value is marked with an underscore "_", and provide an explanation for each slot. 0≤j＜n The corresponding explanation is as follows Where t represents the length of the text, and j represents the j-th slot, i.e., t jLet represent the length of the text in the j-th slot. Then, sentence U can be represented as "s0 means..." Its corresponding value is _; the meaning of s1 is Its corresponding value is _; ...; s n-1 The meaning is Its corresponding value is _;".

[0057] Based on the definition in the above sentence, the relevant information for the m-th slot at the current moment is as follows:

[0058] Input text: O\nSentence to be filled: U\nI

[0059] In the formula, I represents the fill-in-the-blank instruction, n represents the total number of slots, O represents the original input text, and U represents the sentence, expressed as:

[0060] The meaning of "s0" is Its corresponding value is _; ...; s m-1 The meaning is Its corresponding value is _; ...; s n-1 The meaning is Its corresponding value is _;

[0061] Where _ represents the position corresponding to the slot value; s n-1 This represents the name of the nth slot, and its corresponding interpretation is as follows: t n-1 This indicates the length of the text in the nth slot.

[0062] Let t represent the relevant information of the m-th slot at the current moment. m The model is input, and then slot values ​​are extracted step by step using an autoregressive method to finally generate a completed sentence. The calculation process for generating slot values ​​is shown in steps 120-140:

[0063] Step 120: Based on the relevant information and parameter matrix of the m-th slot at the current time, and the hidden layer representation vector of the decoder at the previous time, determine the attention matrix of the m-th slot.

[0064] The computation process of large-scale language models is an autoregressive computation, where each step depends on the output vector of the previous step. Specifically, this invention determines the attention matrix for the m-th slot using the following formula:

[0065]

[0066] Among them, Atten m This represents the attention matrix corresponding to the m-th slot, o m-1 W is the hidden layer representation vector of the decoder at the previous time step. v W qW k They are matrices v, q, and k, respectively, and d k Indicate i m and o m-1 The dimension, i m Indicates t m The encoding vector matrix, where T denotes transpose. The v matrix, q matrix, and k matrix mentioned here are the three core matrices in the attention mechanism, and are the trained parameter matrices.

[0067] Step 130: Based on the attention matrix of the m-th slot, determine the vocabulary probability distribution corresponding to the m-th slot;

[0068] Specifically, such as Figure 2 As shown, this is achieved through the following two steps:

[0069] Step 1301: Based on the attention matrix of the m-th slot, determine the hidden layer representation vector output at the current time.

[0070] Specifically, the hidden layer representation vector output at the current time step is determined according to the following formula:

[0071]

[0072] Among them, o m Atten represents the hidden layer representation vector output at the current time step. m This represents the attention matrix corresponding to the m-th slot. The parameter matrix represents the trained parameter matrix used for linear transformation, converting the length of the attention output matrix to the vocabulary length. m This represents the hidden layer representation vector of the decoder at the current time step.

[0073] Step 1302: Based on the hidden layer representation vector output at the current time, determine the vocabulary probability distribution corresponding to the m-th slot.

[0074] Specifically, the vocabulary probability distribution corresponding to the m-th slot is determined according to the following formula:

[0075] prob vocb =softmax(o m )

[0076] Among them, prob vocb Represents the probability distribution of the vocabulary, o m This represents the hidden layer representation vector output at the current time step, and its size is consistent with the length of the vocabulary.

[0077] Step 140: Based on the probability distribution of the vocabulary, determine the slot value corresponding to the m-th slot and replace the corresponding fill symbol to fill the completed sentence.

[0078] Specifically, the slot value is determined using the following formula, and the corresponding fill-in symbol is replaced to fill in the completed sentence: y m =max(prob vocb ).

[0079] After traversing all slots in the slot set, the calculation ends, and the completed sentence U is finally generated. final This can be represented as "s0 means..." Its corresponding value is v1; the meaning of s1 is Its corresponding values ​​are v1; ...; s n-1 The meaning is Its corresponding value is v n-1 ;".

[0080] The slot value extraction method based on a large model provided by this invention organizes the slot name and slot value into a sentence during the slot information extraction process. The sentence, which contains an explanation of the meaning of each slot, is concatenated with the original input text and input into the large model. This allows the large model to fully understand the meaning of each slot, thereby accurately extracting the slot value corresponding to the slot and preventing the large model from outputting abnormal slots.

[0081] Example 2

[0082] Embodiment 2 of the present invention provides a slot value extraction device based on the slot meaning of a large model, such as... Figure 3 As shown, the device includes: an information information module 310, an attention matrix module 320, a vocabulary probability distribution module 330, and a slot value module 340. Specifically:

[0083] The relevant information module 310 is used to obtain the relevant information of the m-th slot at the current time. The relevant information is composed of the sentence to be filled, the original input text, and the fill-in instruction. The sentence to be filled consists of the slot name, meaning explanation, and corresponding slot value of the first slot, the slot name, meaning explanation, and corresponding slot value of the second slot, and the slot name, meaning explanation, and corresponding slot value of the third slot. The first slot is all slots before the current time, the second slot is the m-th slot at the current time, and the third slot is all slots after the current time. The slot values ​​corresponding to the second slot and the third slot are replaced with fill-in symbols.

[0084] Attention matrix module 320 is used to determine the attention matrix of the m-th slot based on the relevant information and parameter matrix of the m-th slot at the current time, and the hidden layer representation vector of the decoder at the previous time; wherein, the parameter matrix includes the q matrix, k matrix and v matrix in the attention mechanism;

[0085] The vocabulary probability distribution module 330 is used to determine the vocabulary probability distribution corresponding to the m-th slot based on the attention matrix of the m-th slot.

[0086] The slot value module 340 is used to determine the slot value corresponding to the m-th slot based on the probability distribution of the vocabulary, and replace the corresponding fill symbol to fill the completed sentence.

[0087] Furthermore, specifically, the relevant information module 310 is used for:

[0088] Organize the slot name and slot value into a sentence U using natural language, where the slot value is marked with an underscore "_", and provide an explanation for each slot. 0≤j＜n The corresponding explanation is as follows Where t represents the length of the text, and j represents the j-th slot, i.e., t j Let represent the length of the text in the j-th slot. Then, sentence U can be represented as "s0 means..." Its corresponding value is _; the meaning of s1 is Its corresponding value is _; ...; s n-1 The meaning is Its corresponding value is _;".

[0089] Based on the definition in the above sentence, the relevant information for the m-th slot at the current moment is as follows:

[0090] Input text: O\nSentence to be filled: U\nI

[0091] In the formula, I represents the fill-in-the-blank instruction, n represents the total number of slots, O represents the original input text, and U represents the sentence, expressed as:

[0092] The meaning of "s0" is Its corresponding value is _; ...; s m-1 The meaning is Its corresponding value is _; ...; s n-1 The meaning is Its corresponding value is _;

[0093] Where _ represents the position corresponding to the slot value; s n-1 This represents the name of the nth slot, and its corresponding interpretation is as follows: t n-1 This indicates the length of the text in the nth slot.

[0094] Furthermore, specifically, the attention matrix module 320 determines the attention matrix for the m-th slot using the following formula:

[0095]

[0096] Among them, Atten m This represents the attention matrix corresponding to the m-th slot, o m-1 W is the hidden layer representation vector of the decoder at the previous time step. v W q W k They are matrices v, q, and k, respectively, and d k Indicate i m and o m-1 The dimension, i m This represents the encoded vector matrix of the relevant JSON structure information for the m-th slot at the current moment, where T represents transpose.

[0097] Furthermore, such as Figure 4 As shown, the vocabulary probability distribution module 330 includes a first determining unit 3301 and a second determining unit 3302. Specifically,

[0098] The first determining unit 3301 is used to determine the hidden layer representation vector output at the current time based on the attention matrix of the m-th slot.

[0099] Specifically, the hidden layer representation vector output at the current time step is determined according to the following formula:

[0100]

[0101] Among them, Atten m This represents the attention matrix corresponding to the m-th slot. This represents the parameter matrix, used for linear transformations that convert the length of the attention output matrix to the vocabulary length. m This represents the hidden layer representation vector of the decoder at the current time step, that is, the hidden layer representation vector output at the current time step.

[0102] The second determining unit 302 is used to determine the vocabulary probability distribution corresponding to the m-th slot based on the hidden layer representation vector output at the current time.

[0103] Specifically, the vocabulary probability distribution corresponding to the m-th slot is determined according to the following formula:

[0104] prob vocb =softmax(o m )

[0105] Among them, prob vocb Represents the probability distribution of the vocabulary, o m This represents the hidden layer representation vector output at the current time step, and its size is consistent with the length of the vocabulary.

[0106] The apparatus provided in Embodiment 2 of the present invention can execute the method steps in Embodiment 1 of the above method. Its implementation principle and technical effect are similar, and will not be described again here.

[0107] It should be noted that the division of the various modules in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software via processing element calls; they can be fully implemented in hardware; or some modules can be implemented by processing element calls to software, while others are implemented in hardware. For example, a determination module can be a separate processing element, or it can be integrated into a chip within the above device. Alternatively, it can be stored as program code in the memory of the above device, and its function can be called and executed by a processing element of the device. The implementation of other modules is similar. Moreover, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed through integrated logic circuits in the hardware of the processor element or through software instructions.

[0108] For example, these modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more Digital Signal Processors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs). As another example, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a System-on-a-Chip (SOC).

[0109] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The aforementioned computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The aforementioned computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the aforementioned computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, Bluetooth, microwave, etc.) means. The aforementioned computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The aforementioned available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0110] Example 3

[0111] Embodiment 3 of the present invention provides a computer server, such as Figure 5 As shown, it includes: memory, processor, and transceiver;

[0112] The processor is used to couple with the memory, read and execute instructions in the memory to implement any of the large model-based structure enhancement slot value extraction methods provided in Embodiment 1 above.

[0113] The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

[0114] Example 4

[0115] Embodiment 4 of the present invention provides a chip system, such as Figure 6 As shown, it includes a processor and a coupled memory. The memory stores program instructions. When the program instructions stored in the memory are executed by the processor, it implements any of the large model-based structure enhancement slot extraction methods provided in Embodiment 1.

[0116] Example 5

[0117] Embodiment 5 of the present invention provides a computer-readable storage medium, such as... Figure 7As shown, it includes a program or instructions that, when run on a computer, implement any of the large model-based structure enhancement slot extraction methods provided in Embodiment 1.

[0118] The slot value extraction method based on a large model provided by this invention organizes the slot name and slot value into a sentence during the slot information extraction process. The sentence, which contains an explanation of the meaning of each slot, is concatenated with the original input text and input into the large model. This allows the large model to fully understand the meaning of each slot, thereby accurately extracting the slot value corresponding to the slot and preventing the large model from outputting abnormal slots.

[0119] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0120] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0121] The above specific embodiments further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for extracting slot values ​​based on the meaning of slots in a large model, characterized in that, The method includes: Obtain relevant information for the m-th slot at the current time, wherein the relevant information is composed of the sentence to be filled, the original input text, and the fill-in instruction; the sentence to be filled consists of the slot name, meaning explanation, and corresponding slot value of the first slot, the slot name, meaning explanation, and corresponding slot value of the second slot, and the slot name, meaning explanation, and corresponding slot value of the third slot; the first slot is all slots before the current time, and the second slot is the m-th slot at the current time. There are 1 slot, and the third slot is all slots after the current time. The slot values ​​corresponding to the second slot and the third slot are replaced with fill symbols. Based on the current moment, the Using the relevant information and parameter matrix of the slot, as well as the hidden layer representation vector of the decoder from the previous time step, the th slot is determined. Attention matrix for each slot; wherein, the parameter matrix includes the attention mechanism... matrix, Matrix and matrix; Based on the first The attention matrix of the i-th slot determines the i-th The probability distribution of the vocabulary corresponding to each slot; Based on the vocabulary probability distribution, determine the first... The slot value corresponding to each slot is used to replace the corresponding fill-in symbol to fill in the completed sentence; The relevant information is composed of sentences, original input text, and fill-in-the-blank instructions, specifically: Input text: Sentence to be filled: In the formula, This indicates a fill-in-the-blank instruction. Indicates the total number of slots. This represents the original input text. To represent a sentence, it is expressed as: " The meaning is , whose corresponding value is _;…; The meaning is , whose corresponding value is _;…; The meaning is Its corresponding value is _; Where _ represents the position corresponding to the slot value; Indicates the first The slot name for each slot, and its corresponding explanation is as follows: , Indicates the first The length of the text in each slot.

2. The method according to claim 1, characterized in that, The basis of the first The attention matrix of the i-th slot determines the i-th The vocabulary probability distribution corresponding to each slot specifically includes: Based on the first The attention matrix of each slot determines the hidden layer representation vector output at the current time step; Based on the hidden layer representation vector output at the current time, determine the first... The probability distribution of the vocabulary corresponding to each slot.

3. The method according to claim 1, characterized in that, The first one based on the current time Using the relevant information and parameter matrix of the slot, as well as the hidden layer representation vector of the decoder from the previous time step, the th slot is determined. The attention matrix for the i-th slot is specifically determined by the following formula: Attention matrix for each slot: in, Indicates the first Attention matrix corresponding to each slot This is the hidden layer representation vector of the decoder at the previous time step. , They are respectively matrix, Matrix and matrix, express and Dimensions Indicates the current time's... The encoding vector matrix of relevant information for each slot. This indicates transpose.

4. The method according to claim 1, characterized in that, The basis of the first The attention matrix of the i-th slot determines the i-th The vocabulary probability distribution corresponding to the slot is as follows: The probability distribution of the slot is determined according to the following formula. The probability distribution of the vocabulary corresponding to each slot: in, Represents the probability distribution of the vocabulary. Indicates the first Attention matrix corresponding to each slot This represents the parameter matrix, used for linear transformations that convert the length of the attention output matrix to the vocabulary length. This represents the hidden layer representation vector of the decoder at the current time step.

5. A slot value extraction device based on a large model for slot meaning, characterized in that, The device includes: The relevant information module is used to obtain relevant information about the m-th slot at the current time. This relevant information is composed of the sentence to be filled, the original input text, and the fill-in instruction. The sentence to be filled consists of the slot name, meaning explanation, and corresponding slot value of the first slot, the slot name, meaning explanation, and corresponding slot value of the second slot, and the slot name, meaning explanation, and corresponding slot value of the third slot. The first slot represents all slots before the current time, and the second slot represents the m-th slot at the current time. There are 10 slots, and the third slot represents all slots after the current time. The slot values ​​corresponding to the second and third slots are replaced with fill-in symbols. The relevant information is composed of sentences, original input text, and fill-in instructions, specifically: Input text: Sentence to be filled: In the formula, This indicates a fill-in-the-blank instruction. Indicates the total number of slots. This represents the original input text. To represent a sentence, it is expressed as: " The meaning is , whose corresponding value is _;…; The meaning is , whose corresponding value is _;…; The meaning is Its corresponding value is _; Where _ represents the position corresponding to the slot value; Indicates the first The slot name for each slot, and its corresponding explanation is as follows: , Indicates the first The length of the text in each slot; Attention matrix module, used for the current time step Using the relevant information and parameter matrix of the slot, as well as the hidden layer representation vector of the decoder from the previous time step, the th slot is determined. Attention matrix for each slot; wherein, the parameter matrix includes the attention mechanism... matrix, Matrix and matrix; The vocabulary probability distribution module is used to base the probability distribution on the first... The attention matrix of the i-th slot determines the i-th The probability distribution of the vocabulary corresponding to each slot; The slot value module is used to determine the first slot value based on the vocabulary probability distribution. The slot value corresponding to each slot is used to replace the corresponding fill symbol to complete the sentence.

6. The apparatus according to claim 5, characterized in that, The vocabulary probability distribution module includes: The first determining unit is used to determine based on the first... The attention matrix of each slot determines the hidden layer representation vector output at the current time step; The second determining unit is used to determine the first hidden layer representation vector based on the hidden layer representation vector output at the current time. The probability distribution of the vocabulary corresponding to each slot.

7. A computer server, characterized in that, include: Memory, processor, and transceiver; The processor is used to couple with the memory, read and execute instructions in the memory to implement the slot value extraction method based on the slot meaning of a large model as described in any one of claims 1-4; The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

8. A chip system, characterized in that, The system includes a processor coupled to a memory that stores program instructions. When the program instructions stored in the memory are executed by the processor, the system implements the slot value extraction method based on the slot meaning of a large model as described in any one of claims 1-4.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which is executed by a processor as described in any one of claims 1-4, for the method of extracting slot meanings based on a large model.

Images