Position discrimination method and system for multivariate relation auxiliary phrases
Through the multi-relational auxiliary phrase position discrimination method, the UIE backbone network model and multi-task attention network are used to establish the association relationship between task prompts and text fragments, and the positioning network and the discriminating network determine the phrase position, solving the problems of insufficient accuracy and high computational cost in the existing UIE technology, achieving more efficient phrase position judgment and savings in computing resources.
Patent Information
- Application Number
- CN202410505625.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-25
- Publication Date
- 2025-03-25
AI Technical Summary
Existing UIE technologies have shortcomings in evaluating the accuracy of overall phrases, resulting in the possible incorrect phrases after position combinations and the need for repeated calculations during multitasking, resulting in data redundancy and excessive computational costs.
Multivariate-related auxiliary phrase position discrimination method is adopted, including pre-training the UIE backbone network model, establishing the association relationship between task prompts and text fragments through the multi-task attention network, and using the positioning network and the discriminating network to determine the phrase position.
By combining the output of the UIE backbone network model to process the relationship between multi-task prompts and multi-corpus fragments, multi-level label attributes can be better predicted, saving calculation costs, and improving the accuracy of phrase position judgment.
Smart Images
Figure CN119692347A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of corpus analysis and processing, and in particular to a method for extracting commodity keywords through UIE. Background Art
[0002] With the development of Internet technology, UIE, which processes complex corpora, uniformly predicts the start and end positions of phrases. Regardless of the target structure of the information extraction task, UIE can process them in a unified encoding method. This adaptive generation capability enables UIE to be flexibly applied to various complex information extraction tasks, effectively solving the problem of phrase coverage extraction. However, this method may result in incorrect phrases after position combination, mainly due to its shortcomings in evaluating the accuracy of the overall phrase. At the same time, when performing multiple tasks for expected multiple phrases, UIE needs to perform repeated calculations in parallel in the same context, which may lead to redundant data calculations. Especially when the context is too long, the computational cost of this method may become too high. Summary of the invention
[0003] Therefore, it is necessary to provide a low-cost technical solution that can determine the position of phrases in a corpus.
[0004] To achieve the above object, the inventor provides a method for determining the position of a multi-relation auxiliary phrase, comprising the following steps: pre-training a UIE backbone network model, wherein the UIE backbone network model is used to generate first structure data according to task prompts and corpus information;
[0005] The first structure data is input into the multi-task attention network to establish the association relationship between the single task prompt information and the text fragment information.
[0006] The localization network and the discriminant network are then used to determine the position of the phrase corresponding to the task prompt information in the corpus.
[0007] In one embodiment of the present application, a step is further included, wherein the task prompt includes a product name, product brand or product category.
[0008] In one embodiment of the present application, the step of setting the word segmentation positions of the subtasks to increase from 1 is also included, and the step of recording the word segmentation start and end positions of each subtask, as well as the maximum filling number.
[0009] In one embodiment of the present application, the first structure data is of shape [b, s, c1], where b is the number of batches processed at a time, s is the sequence length of all subtasks and context spliced together, and 768 is the feature length.
[0010] In one embodiment of the present application, the positioning network includes two linear layers and a mapping function;
[0011] The positioning network adopts the confidence threshold set to 0.5 for regression iteration during training, and is set to three-segment threshold and extracted in the inference stage, and the three segments 4 are 0.5, 0.3 and 0.1 respectively.
[0012] In one embodiment of the present application, the discriminant network includes a sequence average pooling layer, two stacked linear layers and an activation function.
[0013] The sequence average pooling layer is used to convert the phrase sequence into second structure data, and the stacked linear layer and the activation function are used together to normalize the feature number of the second structure data.
[0014] In one embodiment of the present application, the second structure data is a data stream shape shape[batch_phrase, c2], batch_phrase is a batch phase, and c2 is a second eigenvalue.
[0015] In one embodiment of the present application, in a multi-task attention network, the function key_padding_mask is used to ignore the second half of the maximum padding of the task prompt, and the function attn_mask is used to ignore the padding part of the maximum sequence length of the entire batch.
[0016] A position determination system for multi-relation auxiliary phrases, comprising:
[0017] A UIE backbone network model unit, wherein the UIE backbone network model unit is used to generate first structure data according to the task prompt and corpus information;
[0018] The multi-task attention network unit is used to input the first structure data into the multi-task attention network to establish an association relationship between the single task prompt information and the text segment information.
[0019] A positioning and discrimination unit is used to enable a positioning network and a discrimination network to determine the position of a phrase corresponding to the task prompt information in the corpus.
[0020] In one embodiment of the present application, the multi-task attention network unit specifically uses the function key_padding_mask to ignore the second half of the maximum padding of the task prompt, and uses the function attn_mask to ignore the maximum sequence length padding part of the entire batch;
[0021] The positioning network includes two linear layers and a mapping function, and the discriminant network includes a sequence average pooling layer, two stacked linear layers and an activation function;
[0022] The positioning discrimination unit is used to set the confidence threshold to 0.5 regression iteration when training the positioning network, and set it to three-segment threshold and extract in the inference stage, and the three segments 4 are 0.5, 0.3 and 0.1 respectively, and is also used to convert the phrase sequence into second structure data, and the stacked linear layer and activation function are used together to normalize the feature number of the second structure data.
[0023] Through the above scheme, the present application can process the relationship between multi-task prompts and multi-corpus fragments by combining the output of the UIE backbone network model, can better predict multi-level label attributes, and can better save computing costs.
[0024] The above-mentioned records related to the invention content are only an overview of the technical solution of the present application. In order to enable ordinary technicians in the field to more clearly understand the technical solution of the present application, and then implement it according to the text of the specification and the contents recorded in the drawings, and to make the above-mentioned purpose and other purposes, features and advantages of the present application easier to understand, the following is an explanation in combination with the specific implementation mode and drawings of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] The drawings are only used to illustrate the principles, implementation methods, applications, characteristics and effects of the specific embodiments of the present application and other related contents, and shall not be considered as limitations of the present application.
[0026] Figure 1 It is a flow chart of the method for determining the position of a multi-relation auxiliary phrase according to a specific implementation method;
[0027] Figure 2 A flow chart of a method for determining the position of a multi-relation auxiliary phrase according to another embodiment;
[0028] Figure 3 It is a schematic diagram of a position determination system of a polynary relation auxiliary phrase according to a specific implementation method. DETAILED DESCRIPTION
[0029] In order to explain the technical content, structural features, achieved objectives and effects of the technical solution in detail, the following is a detailed description in conjunction with specific embodiments and accompanying drawings.
[0030] Reference to "embodiment" herein means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present application. The term "embodiment" appearing in various places in the specification does not necessarily refer to the same embodiment, nor does it particularly limit its independence or association with other embodiments. In principle, in the present application, as long as there is no technical contradiction or conflict, the various technical features mentioned in the embodiments can be combined in any way to form a corresponding implementable technical solution.
[0031] Unless otherwise defined, the technical terms used in this document have the same meanings as those generally understood by those skilled in the art to which this application belongs; the use of relevant terms in this document is only for describing specific embodiments and is not intended to limit this application.
[0032] In the description of this application, the term "and / or" is an expression used to describe the logical relationship between objects, indicating that three relationships may exist, for example, A and / or B, which means: A exists, B exists, and A and B exist at the same time. In addition, the character " / " in this article generally indicates that the objects before and after are in an "or" logical relationship.
[0033] In the present application, terms such as “first” and “second” are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship of quantity, priority or sequence between these entities or operations.
[0034] Without further limitations, in this application, the words "include", "comprises", "has" or other similar expressions used in the sentences are intended to cover non-exclusive inclusion. These expressions do not exclude the presence of additional elements in the process, method or product including the elements, so that the process, method or product including a series of elements may include not only those limited elements, but also other elements not explicitly listed, or also include elements inherent to such process, method or product.
[0035] Similar to the understanding in the Examination Guidelines, in this application, expressions such as "greater than", "less than", "exceed" and the like are understood to exclude the number itself; expressions such as "above", "below", "within" and the like are understood to include the number itself. In addition, in the description of the embodiments of this application, "multiple" means more than two (including two), and similar expressions related to "multiple" are also understood in this way, such as "multiple groups", "multiple times", etc., unless otherwise clearly and specifically limited.
[0036] In the description of the embodiments of the present application, the space-related expressions used, such as "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "vertical", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc., indicate the orientation or position relationship based on the orientation or position relationship shown in the specific embodiments or drawings, and are only for the convenience of describing the specific embodiments of the present application or facilitating the understanding of the readers, rather than indicating or implying that the device or component referred to must have a specific position, a specific orientation, or be constructed or operated in a specific orientation, and therefore cannot be understood as limiting the embodiments of the present application.
[0037] Unless otherwise expressly specified or limited, in the description of the embodiments of the present application, the terms such as "install", "connect", "connect", "fix", "set", etc. used should be understood in a broad sense. For example, the "connection" can be a fixed connection, a detachable connection, or an integrated setting; it can be a mechanical connection, an electrical connection, or a communication connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. For technicians in the technical field to which the present application belongs, the specific meanings of the above terms in the embodiments of the present application can be understood according to the specific circumstances.
[0038] In some embodiments, see Figure 1 , is a method for determining the position of a multi-relation auxiliary phrase, comprising the following steps: S100 pre-training a UIE backbone network model, wherein the UIE backbone network model is used to generate first structure data according to task prompts and corpus information; S102 inputting the first structure data generated by the UIE backbone network model into a multi-task attention network, establishing an association relationship between a single task prompt information and text fragment information, and then performing step S104 using a positioning network and a discriminant network to determine the position of the phrase corresponding to the task prompt information in the corpus.
[0039] Among them, the UIE model refers to Unified Information Extraction, which is a universal information extraction model. This model is a unified framework for universal information extraction released in ACL-2022. It can comprehensively realize the unified modeling of tasks such as entity extraction, relationship extraction, event extraction, and sentiment analysis, and enable good migration and generalization capabilities between different tasks. This model can support the extraction of key information without restrictions on industry fields and extraction targets, achieve zero-sample fast cold start, and has excellent small sample fine-tuning capabilities to quickly adapt to specific extraction targets.
[0040] In the technical solution of the present invention, when calling the UIE backbone network model, the ordinary UIE model is modified, and its corresponding task is expanded into multiple subtasks, and the task prompt can also be a task prompt set {prompt1, prompt2, prompt3, ...}. The corpus information is the corpus to be identified, generally text information. In some embodiments of the present application, the task prompt can be a label of text information or corpus information, such as a task prompt of a product introduction text can include a product name, a product brand or a product category.
[0041] For example, the data flow initialization definition is: [cls] extract the product name [sep] extract the brand of the specific product [sep] extract the category of the specific product [sep]...content. The same context content can be visible to multiple subtasks, and each subtask ends with [sep]. Among them, the token_type_ids of the task part are all 0, and the content is all 1. In addition, considering the position position_ids parallelism of each subtask, each token (word segmentation) position of each subtask is set to increase from 1, and the token word segmentation of the content (corpus) starts with the maximum length of the token in the task. The specific data initialization definition is: 0123...123...123... Record the start and end positions of each subtask token, as well as the maximum padding number max_pad_num. The calculation of max_pad_num uses the maximum subtask token length minus the current subtask token length. The backbone network uses the transformers structure to extract the self-attention of the entire token sequence, and can eventually generate the first structure data. Specifically, there is an embedding layer and 24 layers of stacked multi-head self-attention, and the data shape is [b, s, c1], where b is the number of batches processed at a time, s is the sequence length of all tasks and contexts concatenated, and c1 is the feature length. The feature length can be set as needed, for example, it can be set to 768.
[0042] After the data stream is transmitted through the UIE backbone network model, it is sent to the multi-task attention network. The relationships between each subtask prompt and the context content are established and separated. Specifically, each text prompt has a corresponding sequence feature. For each token in the context, the model will check each prompt and perform a weighted summation of the importance of the word token in the context based on each word segmentation token in the prompt to obtain a relationship vector. This relationship vector contains the relationship information between the word token in the context and the text prompt. The main formula for this process is expressed as follows:
[0043] relations, attn_weights=MultiheadAttn(query=batch_prompt_content,key
[0044] =batch_prompt, value=batch_prompt, attn_mask
[0045] =batch_attn_mask, key_padding_mask
[0046] =batch_key_padding_mask)
[0047] Among them, query contains the sequence features of prompts and contexts, and the key and value input shapes are the same, which are extracted from the overall sequence features by previously recording the start, end and fill number of each prompt. In some embodiments, it can be seen that the multi-task attention network can associate multiple subtasks {prompt1, prompt2, prompt3,...} with their corresponding self-attention texts, that is, text fragment information {content1, content2, content3,...}, etc. That is, the association relationship of prompt1-content1, prompt2-content2, prompt3-content3, etc. is output. Then, the positioning network is used to predict the start and end positions of the task prompts in the corresponding text fragment information, and the confidence is output. The discriminant network is also used to discriminate the phrases corresponding to the task prompts, and finally the correct phrases are obtained in the correct position in the corpus. Among them, the phrase is a permutation and combination of one to multiple attributes / labels.
[0048] Through the above scheme, the improved UIE network can be used to perform multi-head attention network processing on multiple serial tasks, and the original relationship between multiple task prompts and corpus text can be transformed into an association relationship between multiple task prompts and multiple text fragments, which can improve the computing efficiency by using the computing performance of multi-threading. At the same time, the association relationship between multiple task prompts and multiple text fragments can improve the accuracy of judging the position of the correct phrase in the corpus.
[0049] In one embodiment of the present application, the step of setting the word segmentation positions of the subtasks to be incremented from 1 is also included, such as Figure 2 As shown, the method further includes step S103 to record the start and end positions of the word segmentation of each subtask, as well as the maximum padding number. In a specific embodiment, the token segmentation of content (corpus) starts with the maximum length of the token in the task, and the specific data initialization definition is: 0123...123...123... Record the start and end positions of each subtask token, as well as the maximum padding number max_pad_num, and the calculation of max_pad_num is to subtract the token length of the current subtask from the token length in the maximum subtask. Through the above scheme, padding can be used to ensure that the sequence features of the subtasks are similar, which is convenient for learning and processing.
[0050] In order to better locate the label, the positioning network includes two linear layers and a mapping function; the data stream coming out of the positioning network is post-processed here, and the post-processing is mainly to extract possible phrases in the sequence. The input of the positioning network is the start, end and corresponding confidence of the possible phrase. The positioning network uses a confidence threshold set to 0.5 for regression iteration during training, and is set to three-segment thresholds and extracted in the inference stage, and the three-segment thresholds are 0.5, 0.3 and 0.1 respectively. For the characteristics of multi-task prompts in parallel, setting a lower threshold can extract more possible phrases, thereby meeting a higher recall rate, allowing more phrase combination situations to enter the next stage for judgment, and achieving the technical effect of improving the fault tolerance rate of this solution.
[0051] In order to better discriminate phrases, the discriminant network includes a sequence average pooling layer, two stacked linear layers and an activation function. The sequence average pooling layer is used to convert the phrase sequence into a second structure data, and the stacked linear layer and the activation function are used together to normalize the feature number of the second structure data. Among them, normalization is to convert the feature number into 1 by two linear layers and a sigmoid, and the value range is between [0, 1]. The closer to 1, the more likely it is to be a correct phrase, so as to achieve the technical effect of finally discriminating the starting position and the ending position of the phrase in the corpus.
[0052] In some further embodiments, the second structure data is a data stream shape shape [batch_phrase, c2], batch_phrase is the batch phase, and c2 is the second eigenvalue. Specifically, after extracting the start and end positions corresponding to each task, the phrase sequence features can be intercepted from the overall sequence features. Each phrase sequence first passes through the average pooling layer and is averaged in the sequence dimension, from the original phrase sequence data shape shape [batch_phrase, sequeeze_phrase, 768] to the data stream shape shape [batch_phrase, 768].
[0053] In some specific embodiments, in a multi-task attention network, the function key_padding_mask is used to ignore the second half of the maximum padding of the task prompt, and the function attn_mask is used to ignore the padding part of the maximum sequence length of the entire batch. This allows the solution to better match the computational requirements of the multi-task attention network and improve computational efficiency.
[0054] In such Figure 3In some of the embodiments shown, a position determination system for multi-relation auxiliary phrases is also introduced, including: a UIE backbone network model unit 300, wherein the UIE backbone network model unit is used to generate first structural data according to task prompts and corpus information; a multi-task attention network unit 302 is used to input the first structural data into a multi-task attention network, establish an association relationship between a single task prompt information and text fragment information, and a positioning determination unit 304, wherein the positioning determination unit is used to enable the positioning network and the determination network to determine the position of the phrase corresponding to the task prompt information in the corpus.
[0055] Through the above scheme, the improved UIE network can be used to perform multi-head attention network processing on multiple serial tasks, and the original relationship between multiple task prompts and corpus text can be transformed into an association relationship between multiple task prompts and multiple text fragments, which can improve the computing efficiency by using the computing performance of multi-threading. At the same time, the association relationship between multiple task prompts and multiple text fragments can improve the accuracy of judging the position of the correct phrase in the corpus.
[0056] In one embodiment of the present application, the multi-task attention network unit specifically uses the function key_padding_mask to ignore the second half of the maximum padding of the task prompt, and uses the function attn_mask to ignore the padding part of the maximum sequence length of the entire batch; the positioning network includes two linear layers Linear and a mapping function, and the discriminant network includes a sequence average pooling layer, two stacked linear layers and an activation function; the positioning discriminant unit is used to use a confidence threshold set to 0.5 regression iteration when training the positioning network, and set to three-segment thresholds and extract in the inference stage, the three segments 4 are 0.5, 0.3 and 0.1 respectively, and is also used to convert the phrase sequence into a second structure data, and the stacked linear layer and the activation function are used together to normalize the feature number of the second structure data. Through the above design, the present application can use the positioning network to extract more possible phrases, to a higher recall rate, and at the same time can better save computing costs.
[0057] It should be noted that, although the above embodiments have been described in this article, the patent protection scope of the present invention is not limited thereby. Therefore, based on the innovative concept of the present invention, changes and modifications made to the embodiments described herein, or equivalent structures or equivalent process changes made using the contents of the present invention specification and drawings, directly or indirectly applying the above technical solutions to other related technical fields, are all included in the patent protection scope of the present invention.
Claims
1. A method for determining the position of a multi-relation auxiliary phrase, characterized in that: The method comprises the following steps: pre-training a UIE backbone network model, wherein the UIE backbone network model is used to generate first structure data according to task prompts and corpus information; The first structure data is input into the multi-task attention network to establish the association relationship between the single task prompt information and the text fragment information. The localization network and the discriminant network are then used to determine the position of the phrase corresponding to the task prompt information in the corpus.
2. The method for determining the position of a multi-relation auxiliary phrase according to claim 1, characterized in that: The method further includes a step in which the task prompt includes a product name, a product brand or a product category.
3. The method for determining the position of a multi-relation auxiliary phrase according to claim 1, characterized in that: The method also includes a step of setting the word segmentation positions of the subtasks to be incremented from 1, and a step of recording the word segmentation start and end positions of each subtask, as well as the maximum filling number.
4. The method for determining the position of a multi-relation auxiliary phrase according to claim 1, characterized in that: The first structure data is of shape [b, s, c1], where b is the number of batches processed at a time, s is the sequence length of all subtasks and context concatenation, and 768 is the feature length.
5. The method for determining the position of a multi-relation auxiliary phrase according to claim 1, characterized in that: The positioning network includes two linear layers and a mapping function; The positioning network adopts the confidence threshold set to 0.5 for regression iteration during training, and is set to three-segment threshold and extracted in the inference stage, and the three segments 4 are 0.5, 0.3 and 0.1 respectively.
6. The method for determining the position of a multi-relation auxiliary phrase according to claim 5, characterized in that: The discriminant network consists of a sequence average pooling layer, two stacked linear layers and an activation function. The sequence average pooling layer is used to convert the phrase sequence into second structure data, and the stacked linear layer and the activation function are used together to normalize the feature number of the second structure data.
7. The method for determining the position of a multi-relation auxiliary phrase according to claim 6, characterized in that: The second structure data is a data stream shape shape[batch_phrase, c2], batch_phrase is the batch phase, and c2 is the second eigenvalue.
8. The method for determining the position of a multi-relation auxiliary phrase according to claim 1, characterized in that: In the multi-task attention network, the function key_padding_mask is used to ignore the second half of the maximum padded sequence of the task prompt, and the function attn_mask is used to ignore the padded part of the maximum sequence length of the entire batch.
9. A position determination system for multi-relation auxiliary phrases, characterized in that: include: A UIE backbone network model unit, wherein the UIE backbone network model unit is used to generate first structure data according to the task prompt and corpus information; The multi-task attention network unit is used to input the first structure data into the multi-task attention network to establish an association relationship between the single task prompt information and the text segment information. A positioning and discrimination unit is used to enable a positioning network and a discrimination network to determine the position of a phrase corresponding to the task prompt information in the corpus.
10. The position determination system of multi-relation auxiliary phrases according to claim 9, characterized in that: The multi-task attention network unit specifically uses the function key_padding_mask to ignore the second half of the maximum padding of the task prompt, and uses the function attn_mask to ignore the padding part of the maximum sequence length of the entire batch; The positioning network includes two linear layers and a mapping function, and the discriminant network includes a sequence average pooling layer, two stacked linear layers and an activation function; The positioning discrimination unit is used to set the confidence threshold to 0.5 regression iteration when training the positioning network, and set it to three-segment threshold and extract in the inference stage, and the three segments 4 are 0.5, 0.3 and 0.1 respectively, and is also used to convert the phrase sequence into second structure data, and the stacked linear layer and activation function are used together to normalize the feature number of the second structure data.