Method and equipment for constructing short entity recall model

Abstract

The invention discloses a short entity recall model construction method and device, a short entity recall model comprising an encoder and a decoder is constructed based on a pre-training language model, the encoder is responsible for inputting semantic codes of a context, the decoder generates entity names in an autoregression mode, and the model parameter scale is related to the size of a vocabulary; after obtaining training data containing an input context and a corresponding target entity name, inputting the training data into the model, and training the model in an autoregression mode to learn a mapping relationship between the input context and the target entity name; configuring a limited decoding strategy to ensure that an entity name generated during model reasoning belongs to a preset candidate entity set, and obtaining a trained model; and inputting the context to be retrieved into the model, generating a corresponding entity name through the model, and completing short entity recall.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to a method and apparatus for constructing a short entity recall model. Background Technology

[0002] Short entity retrieval is a core prerequisite for natural language processing tasks such as information retrieval, intelligent question answering, and entity linking. Due to their short text length, semantic focus, and clear referentiality, short entities are widely used in fields such as healthcare, e-commerce, and law. With the continuous expansion of knowledge bases in various fields, the number of short entities is growing explosively. How to quickly and accurately retrieve target entities relevant to the input context from massive datasets has become a critical requirement in these fields, directly impacting the efficiency and accuracy of downstream operations.

[0003] Current mainstream short entity recall technologies employ text vector representation and similarity matching. Based on pre-trained models such as BERT and RoBERTa, the input context and short entities are converted into dense vectors. During the training phase, the vector similarity between positive and negative sample pairs is optimized through contrastive learning. During the inference phase, the similarity between the context vector to be retrieved and the full set of entity vectors is calculated, and candidate entities are returned in order of ranking. Some solutions add a re-ranking module to improve accuracy.

[0004] Existing recall models place high demands on both the quality and quantity of samples during training. However, selecting positive and negative samples in large-scale, complex knowledge bases is challenging, and manual annotation is inefficient. Furthermore, poor sample consistency and subjective differences among annotators introduce noise into the training data, which can interfere with model learning. Summary of the Invention

[0005] To address the aforementioned issues, this application provides a method and apparatus for constructing a short entity recall model.

[0006] The embodiments of this application disclose the following technical solutions:

[0007] In a first aspect, embodiments of this application provide a method for constructing a short entity recall model, the method comprising:

[0008] A short entity recall model is constructed based on a pre-trained language model, consisting of an encoder and a decoder architecture. The encoder is used to semantically encode the input context, and the decoder is used to generate entity names in an autoregressive manner. The parameter size of the short entity recall model is related to the size of the vocabulary.

[0009] After acquiring the training data, the training data is input into the short entity recall model to train the short entity recall model in an autoregressive manner to learn the mapping relationship for generating target entity names based on the input context; the training data includes the input context and the corresponding target entity names;

[0010] Configure a restricted decoding strategy so that the entity names generated during the inference of the short entity recall model belong to a preset candidate entity set, and obtain the trained short entity recall model;

[0011] The context to be retrieved is input into the trained short entity recall model, and the entity name corresponding to the context to be retrieved is generated by the trained short entity recall model to achieve short entity recall.

[0012] In one possible implementation, the process of acquiring the training data includes:

[0013] Extract the entity's metadata and the entity's context in real short text corpora from a pre-set knowledge base; wherein, the metadata includes the entity's standard name, definition description, and application scenario description;

[0014] Semantic role labeling technology is used to identify the semantic components of the entity in the context and filter out redundant statements that are irrelevant to the entity, so as to ensure that the input context focuses on the key association information of the short entity;

[0015] The semantic association confidence between the context and the entity is calculated using an entity linking tool. Entity samples with confidence scores higher than a preset value are retained to construct training data that includes the input context and the target entity name; the input context and the target entity name correspond one-to-one.

[0016] In one possible implementation, training the short entity recall model in an autoregressive manner to learn the mapping relationship for generating target entity names based on the input context includes:

[0017] For each of the training data, the input context in the training data is input into the encoder of the short entity recall model, and the encoder performs semantic encoding on the input context to output the context semantic features corresponding to the input context.

[0018] The contextual semantic features are fed into the decoder of the short entity recall model. Based on the contextual semantic features, the decoder uses sequential autoregression to determine the predicted token sequence of the target entity name in units of tokens.

[0019] Calculate the loss value between the predicted token sequence and the real token sequence corresponding to the target entity name in the training data;

[0020] Based on the loss value, the network parameters of the short entity recall model are updated through backpropagation algorithm to optimize the learning effect of the short entity recall model on the mapping relationship between the input context and the target entity name.

[0021] In one possible implementation, the training process further includes:

[0022] Before training, all the completed training data is randomly divided into a training set and a validation set according to a preset ratio; the training set is used for iterative updates of model parameters, and the validation set is used to monitor the model training effect.

[0023] During the training process of the short entity recall model, after each preset round of training set traversal learning is completed, parameter updates are paused and the model verification process is started.

[0024] The input context in the validation set is sequentially input into the short entity recall model in the current training phase to obtain the corresponding predicted entity name generated by the short entity recall model;

[0025] Calculate the matching accuracy between each predicted entity name and the corresponding real target entity name in the validation set, and calculate the F1 score of the validation set;

[0026] If the F1 score of the validation set reaches the preset target threshold, the model training effect is deemed satisfactory, and training can be terminated. If the F1 score of the validation set does not reach the preset target threshold, the types of unmatched samples are analyzed, and after adjusting the training parameters, iterative learning of the training set is resumed until the validation metric is met.

[0027] Secondly, embodiments of this application disclose an apparatus for constructing a short entity recall model, the apparatus comprising:

[0028] A building module is used to construct a short entity recall model based on a pre-trained language model, consisting of an encoder and a decoder architecture. The encoder is used to semantically encode the input context, and the decoder is used to generate entity names in an autoregressive manner. The parameter size of the short entity recall model is related to the size of the vocabulary.

[0029] The training module is used to input the training data into the short entity recall model after acquiring the training data, and train the short entity recall model in an autoregressive manner to learn the mapping relationship for generating target entity names based on the input context; the training data includes the input context and the corresponding target entity names;

[0030] The configuration module is used to configure a restricted decoding strategy so that the entity names generated during the inference of the short entity recall model belong to a preset candidate entity set, thereby obtaining a trained short entity recall model.

[0031] The recall module is used to input the context to be retrieved into the trained short entity recall model, and generate the entity name corresponding to the context to be retrieved through the trained short entity recall model, thereby realizing short entity recall.

[0032] In one possible implementation, the device further includes an acquisition module;

[0033] The acquisition module is used to extract metadata of entities and the context of the entities in real short text corpora from a preset knowledge base; wherein, the metadata includes the entity's standard name, definition description, and application scenario description; semantic role labeling technology is used to identify the semantic components of the entities labeled in the context, and redundant statements unrelated to the entities are filtered to ensure that the input context focuses on the key association information of the short entities; the semantic association confidence between the context and the entities is calculated through an entity linking tool, and entity samples with confidence scores higher than a preset value are retained to construct training data including the input context and the target entity name; the input context and the target entity name correspond one-to-one.

[0034] In one possible implementation, the training module is specifically used to: input the input context from the training data into the encoder of the short entity recall model for each training data set; semantically encode the input context using the encoder to output context semantic features corresponding to the input context; input the context semantic features into the decoder of the short entity recall model; determine the predicted token sequence of the target entity name by sequential autoregression based on the context semantic features, using tokens as units; calculate the loss value between the predicted token sequence and the real token sequence corresponding to the target entity name in the training data; and update the network parameters of the short entity recall model using a backpropagation algorithm based on the loss value to optimize the learning effect of the short entity recall model on the mapping relationship between the input context and the target entity name.

[0035] In one possible implementation, the training module is further configured to randomly divide all the constructed training data into a training set and a validation set according to a preset ratio before training; wherein, the training set is used for iterative updating of model parameters, and the validation set is used for monitoring the model training effect; during the training process of the short entity recall model, after completing a preset number of training set traversal learning rounds, parameter updates are paused and the model validation process is started; the input context in the validation set is sequentially input into the short entity recall model in the current training stage to obtain the corresponding predicted entity names generated by the short entity recall model; the matching accuracy of each predicted entity name with the corresponding real target entity name in the validation set is calculated, and the F1 value of the validation set is calculated; if the F1 value of the validation set reaches a preset target threshold, the model training effect is determined to be satisfactory, and training can be terminated; if the F1 value of the validation set does not reach the preset target threshold, the type of unmatched samples is analyzed, and after adjusting the training parameters, the iteration learning of the training set is continued until the validation index is satisfactory.

[0036] Thirdly, embodiments of this application disclose a control device, including a processor and a memory, wherein the memory is used to store programs, instructions or code, and the processor is used to execute the programs, instructions or code in the memory to complete the method for constructing a short entity recall model as described in any of the first aspects.

[0037] Fourthly, embodiments of this application disclose a computer-readable storage medium, characterized in that it stores a computer program, which is loaded by a processor to execute the method for constructing a short entity recall model as described in any of the first aspects.

[0038] The method for constructing a short entity recall model provided in this application embodiment involves building a short entity recall model containing an encoder and a decoder based on a pre-trained language model. The encoder is responsible for semantic encoding of the input context, and the decoder generates entity names in an autoregressive manner. The model parameter size is related to the vocabulary size. After obtaining training data containing the input context and the corresponding target entity names, the data is input into the model, and the model is trained in an autoregressive manner to learn the mapping relationship between the input context and the target entity names. A restricted decoding strategy is configured to ensure that the entity names generated by the model during inference belong to a preset candidate entity set, thus obtaining a trained model. The context to be retrieved is input into the model, and the model generates the corresponding entity names to complete short entity recall. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 A flowchart illustrating a method for constructing a short entity recall model provided in an embodiment of this application;

[0041] Figure 2 A closed-loop workflow diagram for instruction data construction and recall model training provided in this application embodiment;

[0042] Figure 3 This is a schematic diagram of a device for constructing a short entity recall model, provided in an embodiment of this application. Detailed Implementation

[0043] As described above, existing short entity recall technologies face several unavoidable technical problems in practical applications: First, sample construction costs are high. Existing models have stringent requirements for the quality and quantity of training data. When the target knowledge base is large (e.g., millions or tens of millions of entities) and complex in structure, the selection of positive and negative sample pairs becomes extremely difficult, and the efficiency of manually constructing sample pairs is low, making it difficult to meet the needs of large-scale training. Second, sample consistency is poor. The process of manually selecting positive and negative sample pairs is easily affected by subjective factors such as differences in the cognitive abilities and working conditions of the annotators. Different annotators have inconsistent judgment logic on the relevance of samples, leading to inconsistent training data. The data contains a large amount of noise, which interferes with the model's learning performance; thirdly, the vector storage and computation pressure is high. As the number of short entities in the knowledge base increases, the scale of entity vectors that need to be stored expands accordingly, resulting in a linear increase in memory usage. Moreover, the vector similarity calculation during inference requires traversing the entire set of entity vectors, which is time-consuming and makes it difficult to meet the business requirements of low-latency recall; fourthly, the cold start problem is prominent. For newly added short entities in the knowledge base, the corresponding semantic vectors need to be regenerated and the vector library updated. In some scenarios, additional labeled samples are also required to perform secondary training on the model, resulting in low adaptation efficiency and an inability to quickly achieve effective recall of newly added entities.

[0044] To address this technical problem, this application provides a method and apparatus for constructing a short entity recall model. This method transforms the short entity recall task into an entity name generation task through an encoder-decoder architecture and an autoregressive generation mode. It eliminates the need for traditional vector similarity matching, and the model parameter size is only related to the vocabulary size, effectively avoiding the storage and computational pressure caused by the growth in the number of entities. A restricted decoding strategy ensures the effectiveness and relevance of the generated entity names, avoiding irrelevant output. By training the model to learn the direct mapping relationship between context and entity names, the training process is simplified without the need for complex positive-negative sample pair construction, while simultaneously improving the targeting and efficiency of short entity recall, thus meeting the core needs of massive short entity recall scenarios.

[0045] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0046] See Figure 1 , Figure 1 A flowchart illustrating a method for constructing a short entity recall model provided in this application embodiment is shown. The method includes:

[0047] S11: A short entity recall model based on an encoder and decoder architecture built from a pre-trained language model.

[0048] The encoder is used to semantically encode the input context, and the decoder is used to generate entity names in an autoregressive manner. The parameter size of the short entity recall model is related to the vocabulary size.

[0049] In this embodiment, the BART-base model can be selected as the pre-trained language model to construct the encoder and decoder architecture of the short entity recall model. The encoder can employ a 6-layer bidirectional Transformer structure, with each layer containing a multi-head attention mechanism, a feedforward neural network, and a layer normalization module. The input context is converted into fixed-dimensional word vectors via word embedding layers, and semantic information is extracted layer by layer through the bidirectional Transformer layers, ultimately outputting a vector sequence containing complete contextual semantics.

[0050] The decoder uses a 6-layer unidirectional Transformer structure, which also includes a multi-head attention mechanism and a feedforward neural network. It establishes a connection with the semantic vector sequence output by the encoder through a cross-attention layer to ensure that the generated entity name is consistent with the semantics of the input context.

[0051] The parameters of the short entity recall model are directly reused from the pre-trained parameters of the BART-base model. The vocabulary adopts the native vocabulary of the short entity recall model, which can include 32,000 tokens. The overall parameter size of the short entity recall model is about 120 million. This parameter size is only related to the number of tokens in the native vocabulary. Even if the number of entities in the candidate entity set expands from hundreds of thousands to tens of millions, the model parameters do not need to be adjusted and remain at a fixed size.

[0052] This application's embodiments transform short entity recall into a semantically driven generation task by constructing an encoder-decoder architecture model. This allows the model to directly learn the mapping relationship between input context and entity names, improving the semantic relevance of the recall. The reuse of pre-trained language models through transfer learning reduces the computational cost of training the model from scratch and shortens the training cycle. Furthermore, the design of binding the model parameter size to the vocabulary size completely solves the problem of surging storage and computational pressure caused by the growth of the number of entities in traditional models, providing lightweight architectural support for massive short entity recall scenarios.

[0053] S12: After obtaining the training data, input the training data into the short entity recall model and train the short entity recall model in an autoregressive manner to learn the mapping relationship of generating target entity names based on the input context.

[0054] The training data includes the input context and the corresponding target entity names. The input context is natural language text related to the short entity, typically short text. The input context includes entity mentions and scene background, such as a patient experiencing sudden chest pain in a medical setting, or purchasing wireless noise-canceling headphones in an e-commerce setting. The target entity names are short, standard entity names that semantically match the input context and are derived from a pre-defined candidate entity set.

[0055] In this embodiment, the training data is acquired based on a pre-defined short entity knowledge base, such as a medical ICD-10 encoding library. Each piece of training data satisfies the requirement of a strong semantic association between the input context and the target entity name. For example, the input context is "acute bronchitis in children," and the corresponding target entity name is "acute bronchitis." After acquiring the training data, it is input into the short entity recall model in batches. The input context is first passed to the encoder for semantic encoding, generating a semantic vector sequence containing scene features and entity association information.

[0056] The decoder initiates autoregressive training based on this semantic vector sequence: starting with the initial token of the target entity name, it sequentially predicts each subsequent token, with each token's generation depending on the preceding generated results and the encoder's semantic vector. During training, a cross-entropy loss function is used to calculate the loss between the token sequence generated by the decoder and the actual token sequence corresponding to the target entity name in the training data. The loss value is then backpropagated to each layer of the encoder and decoder network using the Adam optimizer, iteratively updating the model parameters.

[0057] This application's embodiments train the model through a direct mapping between the input context and the target entity name, eliminating the need to construct complex positive and negative sample pairs. This simplifies the training process and avoids the problems of high sample construction costs and poor consistency in traditional contrastive learning. The autoregressive training method enables the model to optimize semantic matching accuracy token-by-token, strengthening the semantic correlation between the input context and the target entity name, and improving the model's recall accuracy. Simultaneously, the training process reuses the semantic foundation of the pre-trained model, optimizing the mapping relationship only through transfer learning, reducing the computational overhead and data dependency of model training, and ensuring that the model can quickly adapt to short entity recall tasks in different domains.

[0058] During the training process in step S12, the quality of the training data directly determines the model's learning effect on the mapping relationship between the input context and the target entity name. The core of high-quality training data lies in the strong semantic correlation between the input context and the target entity name. To ensure the effectiveness and reliability of the training data, this application embodiment provides a targeted method for obtaining training data.

[0059] In one possible implementation, the process of acquiring the training data includes:

[0060] Extract the entity's metadata and the entity's context in real short text corpora from a pre-set knowledge base; wherein, the metadata includes the entity's standard name, definition description, and application scenario description;

[0061] Semantic role labeling technology is used to identify the semantic components of the entity in the context and filter out redundant statements that are irrelevant to the entity, so as to ensure that the input context focuses on the key association information of the short entity;

[0062] The semantic association confidence between the context and the entity is calculated using an entity linking tool. Entity samples with confidence scores higher than a preset value are retained to construct training data that includes the input context and the target entity name; the input context and the target entity name correspond one-to-one.

[0063] This application embodiment extracts metadata and corresponding real short text corpus context for each entity in batches from a preset short entity knowledge base. The metadata must fully include the entity's standard name, concise definition description, and application scenario description. The following text is derived from real-world corpus of the entity in actual applications, such as user search logs or consultation dialogue fragments, ensuring that the text has scenario authenticity and semantic representativeness.

[0064] Then, semantic role labeling technology is used to parse the extracted context, identifying the core semantic components directly related to the entity, including the entity's corresponding subject, attributes, actions, and associated scenarios. Simultaneously, redundant statements unrelated to the entity are filtered out, ensuring that the processed input context focuses on the entity's key relational information and avoiding redundant information interfering with model learning.

[0065] Finally, using a mature entity linking tool, the semantic association confidence between the purified context and the corresponding entity metadata is calculated. This confidence is quantified using cosine similarity. A pre-set confidence threshold is set, and only entity samples with confidence scores higher than this threshold are retained, while noisy data with weak semantic associations are removed. The validated context is used as the input context, and the corresponding standard entity name is used as the target entity name to construct a one-to-one correspondence of training data, forming a high-quality model training input set.

[0066] To accurately learn the mapping relationship between the input context and the target entity name, this application provides a specific implementation method for the autoregressive training process.

[0067] In one possible implementation, training the short entity recall model in an autoregressive manner to learn the mapping relationship for generating target entity names based on the input context includes:

[0068] For each of the training data, the input context in the training data is input into the encoder of the short entity recall model, and the encoder performs semantic encoding on the input context to output the context semantic features corresponding to the input context.

[0069] The contextual semantic features are fed into the decoder of the short entity recall model. Based on the contextual semantic features, the decoder uses sequential autoregression to determine the predicted token sequence of the target entity name in units of tokens.

[0070] Calculate the loss value between the predicted token sequence and the real token sequence corresponding to the target entity name in the training data;

[0071] Based on the loss value, the network parameters of the short entity recall model are updated through backpropagation algorithm to optimize the learning effect of the short entity recall model on the mapping relationship between the input context and the target entity name.

[0072] First, for each training data point, the input context is fed into the encoder of the short entity recall model. The encoder converts the input context into fixed-dimensional word vectors through a word embedding layer, and then extracts the core semantic information of the input context layer by layer through a multi-head attention mechanism of a bidirectional Transformer layer and a feedforward neural network, finally outputting a sequence of context semantic feature vectors containing complete semantics.

[0073] Then, the contextual semantic features output by the encoder are fed into the decoder. The decoder captures the fine-grained associations with these semantic features through a cross-attention layer, initiating an autoregressive generation process. The decoder predicts each lexical unit of the target entity name sequentially, token by token. For example, when the target entity name is "acute pharyngitis", the decoder first predicts the first token "acute" based on the semantic features, then combines "acute" with the contextual semantic features to predict the second token "sexual", and so on, until a complete predicted token sequence "[acute, sexual, pharyngitis]" is generated. The generation of each token depends on the semantic association between the previously generated tokens and the input context.

[0074] Finally, the cross-entropy loss function is used to calculate the loss value between the predicted token sequence output by the decoder and the real token sequence corresponding to the target entity name in the training data. This loss value quantifies the degree of deviation between the prediction result and the actual result. Based on this loss value, gradient information is propagated backward along the model network layers using the backpropagation algorithm to adaptively update the network parameters of the encoder and decoder. This allows the model to gradually adjust its semantic mapping logic and optimize the learning effect of the association between the input context and the target entity name.

[0075] In this embodiment, the encoder's semantic encoding process fully leverages the core semantic features of the input context, providing a solid foundation for subsequent token prediction. The decoder generates a token sequence sequentially via autoregression, conforming to the character composition rules of short entity names, ensuring the consistency and accuracy of the generated results. The calculation of the loss value and backpropagation enable targeted optimization of model parameters, allowing the model to quickly correct prediction biases and gradually improve semantic mapping accuracy. Furthermore, optimization is performed on individual training data points, followed by batch iterations to improve the overall model performance. This ensures the learning effect of each set of training data and achieves global semantic pattern extraction through parameter sharing, laying the core capability foundation for the model's subsequent accurate recall of short entities.

[0076] To avoid overfitting and training bias during model training and to ensure that the short entity recall model can maintain stable short entity recall performance on unseen data, this embodiment adds a training set and validation set partitioning and iterative validation process on the basis of autoregressive training. Through phased performance monitoring and parameter adjustment, the model training effect can be accurately controlled.

[0077] In one possible implementation, the training process further includes:

[0078] Before training, all the completed training data is randomly divided into a training set and a validation set according to a preset ratio; the training set is used for iterative updates of model parameters, and the validation set is used to monitor the model training effect.

[0079] During the training process of the short entity recall model, after each preset round of training set traversal learning is completed, parameter updates are paused and the model verification process is started.

[0080] The input context in the validation set is sequentially input into the short entity recall model in the current training phase to obtain the corresponding predicted entity name generated by the short entity recall model;

[0081] Calculate the matching accuracy between each predicted entity name and the corresponding real target entity name in the validation set, and calculate the F1 score of the validation set;

[0082] If the F1 score of the validation set reaches the preset target threshold, the model training effect is deemed satisfactory, and training can be terminated. If the F1 score of the validation set does not reach the preset target threshold, the types of unmatched samples are analyzed, and after adjusting the training parameters, iterative learning of the training set is resumed until the validation metric is met.

[0083] Before formal model training, all training data constructed using the methods described above is randomly divided into training and validation sets according to a preset ratio. The training set has a higher proportion and is used for iterative updates of model parameters and deep learning of semantic mapping relationships. The validation set is independent of the training set, and its data distribution is consistent with that of the training set. It is used solely to objectively evaluate the model's training performance and avoid performance misjudgments due to data leakage.

[0084] During model training, a preset number of training epochs is set as the validation interval. After completing each preset number of training epochs, the parameter update process is paused, and model validation is initiated. For example, when the preset number of epochs is 5, the model stops updating parameters and enters the validation phase after completing 5 iterations of learning the full training set, ensuring that the validation results reflect the model's true performance in the current training phase.

[0085] Next, all input contexts from the validation set are sequentially input into the short entity recall model during the current training phase. The model generates predicted entity names for each input context through encoder encoding and decoder autoregression. Subsequently, each predicted entity name is compared with the corresponding real target entity name in the validation set to calculate the matching result for a single sample. Based on the matching results of all samples, the F1 score of the validation set is calculated. The F1 score serves as the core evaluation metric, comprehensively reflecting the model's recall accuracy and completeness.

[0086] If the F1 score on the current validation set reaches the preset target threshold, it indicates that the model has fully learned the mapping relationship between the input context and the target entity name, and the generalization ability meets the standard. The training effect is deemed satisfactory, and the overall training process can be terminated. If the F1 score does not reach the target threshold, the types of unmatched samples are analyzed, such as scene adaptation bias, semantic association omissions, and lack of coverage of short entity name variants. The training parameters are adjusted accordingly. After adjustment, the training phase is returned, and iterative learning on the training set continues until the F1 score reaches the preset threshold in subsequent validations.

[0087] S13: Configure a restricted decoding strategy so that the entity names generated during the inference of the short entity recall model belong to a preset candidate entity set, thereby obtaining a trained short entity recall model.

[0088] The constrained decoding strategy is a set of restrictive generation rules for the model inference stage. It is used to limit the output range of the decoder, ensure that the generated entity names meet the preset requirements, and avoid irrelevant or invalid results.

[0089] The preset candidate entity set comes from the short entity knowledge base of the target application scenario, containing all short entities that may need to be recalled, such as the set of disease names in the medical scenario and the set of product categories in the e-commerce scenario. It is the only selectable range of the model's generated results.

[0090] In the embodiment of the present application, the core of configuring the restricted decoding strategy is to strictly limit the model generation scope within a preset candidate entity set. First, preprocess all short entity names in the preset candidate entity set. Convert each entity name into a corresponding token sequence according to the word segmentation rules of the short entity recall model, establish a one-to-one mapping dictionary between the entity name and the token sequence, and perform deduplication processing on the token sequence to ensure that each token combination corresponds to only one entity name.

[0091] Subsequently, construct a prefix tree index structure based on this mapping dictionary. Insert the token sequences of all entity names into the prefix tree layer by layer in alphabetical order, so that each node of the prefix tree corresponds to a token, and the path corresponds to the complete entity name token sequence. During the decoder generation process, only allow selecting the next token from the candidate token subset corresponding to the current prefix tree node at each step. For example, when the currently generated token is "urgent", only the tokens corresponding to the children nodes of the "urgent" node in the prefix tree (such as "condition", "abdomen", etc.) can be selected, avoiding generating invalid tokens outside the candidate entity set.

[0092] At the same time, optimize the generation path in combination with the beam search algorithm, set the beam width to 5, that is, retain the 5 candidate token sequences with the highest similarity at each generation step, and preferentially select the path with closer semantic association by calculating the matching degree between each candidate sequence and the input context semantic features until a complete entity name is generated or the preset maximum generation length is reached. Through the above combination of rules, a complete restricted decoding strategy is formed, enabling the model to only output entity names within the preset candidate entity set during inference.

[0093] S14: Input the context to be retrieved into the trained short entity recall model, and generate the entity name corresponding to the context to be retrieved through the trained short entity recall model to achieve short entity recall.

[0094] First, preprocess the context to be retrieved, and process the context to be retrieved according to the unified standard during model training to ensure that its format is consistent with the input context in the training data, avoiding the impact of format differences on the semantic encoding effect. For example, truncate the ultra-long text to 512 tokens and pad and complete the short text.

[0095] Subsequently, the preprocessed context to be retrieved is input into the trained short entity recall model. The encoder first performs semantic encoding, extracts core semantic features through a bidirectional Transformer layer, and outputs a semantic vector sequence. Based on this semantic vector sequence, the decoder initiates an autoregressive generation process while strictly adhering to a constrained decoding strategy. Under the constraint of the pre-tree index, a candidate subset is selected token by token, and a bundle search algorithm is used to retain the optimal generation path, ensuring that the generation process is efficient and does not deviate from the range of the candidate entity set.

[0096] Finally, when the decoder generates a complete entity name or reaches the preset maximum generation length and matches a valid entity, the generation process terminates, and the entity name is output as the recall result corresponding to the context to be retrieved, thus completing the short entity recall.

[0097] This application also provides a closed-loop workflow diagram for instruction data construction and recall model training, such as... Figure 2 As shown.

[0098] Starting from the query set and target database, preliminary candidate data is generated through two parallel approaches: establishing a recall source based on statistical methods and obtaining candidates using a recall model. This candidate data is then fed into a large model judgment module, which distinguishes and labels positive and negative samples in the candidate data, thereby constructing a dataset containing positive and negative sample pairs. This dataset will subsequently serve as a core component of the training set for iterative training of the short entity recall model.

[0099] Simultaneously, the process also includes a branch for instruction optimization. The large model instruction module rewrites the initial instructions to generate candidate instructions. These candidate instructions, along with the candidate data output by the short entity recall model, enter the manual intervention stage, where humans screen and correct the validity and accuracy of the data. Finally, the results of the manual intervention flow back to the candidate acquisition module using the short entity model, supplementing and updating the data source, thus forming a complete closed loop of data generation, model training, instruction optimization, and data feedback, ensuring the performance iteration of the recall model and the continuous improvement of instruction data.

[0100] This application also provides an apparatus for constructing a short entity recall model, such as... Figure 3 As shown, the device includes:

[0101] Module 301 is used to build a short entity recall model based on a pre-trained language model, which consists of an encoder and a decoder architecture. The encoder is used to semantically encode the input context, and the decoder is used to generate entity names in an autoregressive manner. The parameter size of the short entity recall model is related to the size of the vocabulary.

[0102] The training module 302 is used to input the training data into the short entity recall model after acquiring the training data, and train the short entity recall model in an autoregressive manner to learn the mapping relationship for generating target entity names based on the input context; the training data includes the input context and the corresponding target entity names;

[0103] Configuration module 303 is used to configure a restricted decoding strategy so that the entity names generated during the inference of the short entity recall model belong to a preset candidate entity set, thereby obtaining a trained short entity recall model.

[0104] The recall module 304 is used to input the context to be retrieved into the trained short entity recall model, and generate the entity name corresponding to the context to be retrieved through the trained short entity recall model to realize short entity recall.

[0105] In one possible implementation, the device further includes an acquisition module;

[0106] The acquisition module is used to extract metadata of entities and the context of the entities in real short text corpora from a preset knowledge base; wherein, the metadata includes the entity's standard name, definition description, and application scenario description; semantic role labeling technology is used to identify the semantic components of the entities labeled in the context, and redundant statements unrelated to the entities are filtered to ensure that the input context focuses on the key association information of the short entities; the semantic association confidence between the context and the entities is calculated through an entity linking tool, and entity samples with confidence scores higher than a preset value are retained to construct training data including the input context and the target entity name; the input context and the target entity name correspond one-to-one.

[0107] In one possible implementation, the training module 302 is specifically configured to, for each training data, input the input context from the training data into the encoder of the short entity recall model, perform semantic encoding on the input context through the encoder, and output the context semantic features corresponding to the input context; input the context semantic features into the decoder of the short entity recall model, and determine the predicted token sequence of the target entity name by sequential autoregression based on the context semantic features, using tokens as units; calculate the loss value between the predicted token sequence and the real token sequence corresponding to the target entity name in the training data; and update the network parameters of the short entity recall model through a backpropagation algorithm based on the loss value to optimize the learning effect of the short entity recall model on the mapping relationship between the input context and the target entity name.

[0108] In one possible implementation, the training module 302 is further configured to randomly divide all the constructed training data into a training set and a validation set according to a preset ratio before training; wherein, the training set is used for iterative updating of model parameters, and the validation set is used for monitoring the model training effect; during the training process of the short entity recall model, after completing a preset number of training set traversal learning rounds, parameter updates are paused and the model validation process is started; the input context in the validation set is sequentially input into the short entity recall model in the current training stage to obtain the corresponding predicted entity names generated by the short entity recall model; the matching accuracy of each predicted entity name with the corresponding real target entity name in the validation set is calculated, and the F1 value of the validation set is calculated; if the F1 value of the validation set reaches a preset target threshold, the model training effect is determined to be satisfactory, and training can be terminated; if the F1 value of the validation set does not reach the preset target threshold, the type of unmatched samples is analyzed, and after adjusting the training parameters, the iterative learning of the training set is continued until the validation index is satisfactory.

[0109] This application also provides a control device. The control device may include a memory and a processor. The processor is used to execute the method for constructing the short entity recall model described in any of the above embodiments. The memory may be random access memory (RAM), flash memory, read-only memory (ROM), non-volatile read-only memory (EPROM), registers, hard disk, removable disk, etc.

[0110] Memory can store computer instructions, which, when executed by a processor, can be used to implement methods for constructing short entity recall models. Memory can also store data.

[0111] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape) or a semiconductor medium (e.g., solid-state disk (SSD)).

[0112] This application also provides a readable storage medium for storing the methods provided in the above embodiments. For example, RAM, flash memory, ROM, EPROM, registers, hard disk, removable disk, or any other form of storage medium in the art.

[0113] In the embodiments of this application, the terms "first" and "second" (if they exist) are used only as name identifiers and do not represent the order of first and second.

[0114] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. Regarding the methods disclosed in the embodiments, since they correspond to the product embodiments disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the description of the product embodiments.

[0115] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for constructing a short entity recall model, characterized in that, The method includes: A short entity recall model is constructed based on a pre-trained language model, consisting of an encoder and a decoder architecture. The encoder is used to semantically encode the input context, and the decoder is used to generate entity names in an autoregressive manner. The parameter size of the short entity recall model is related to the size of the vocabulary. After acquiring the training data, the training data is input into the short entity recall model to train the short entity recall model in an autoregressive manner to learn the mapping relationship for generating target entity names based on the input context; the training data includes the input context and the corresponding target entity names; Configure a restricted decoding strategy so that the entity names generated during the inference of the short entity recall model belong to a preset candidate entity set, and obtain the trained short entity recall model; The context to be retrieved is input into the trained short entity recall model, and the entity name corresponding to the context to be retrieved is generated by the trained short entity recall model to achieve short entity recall.

2. The method according to claim 1, characterized in that, The process of acquiring the training data includes: Extract the entity's metadata and the entity's context in real short text corpora from a pre-set knowledge base; wherein, the metadata includes the entity's standard name, definition description, and application scenario description; Semantic role labeling technology is used to identify the semantic components of the entity in the context and filter out redundant statements that are irrelevant to the entity, so as to ensure that the input context focuses on the key association information of the short entity; The semantic association confidence between the context and the entity is calculated using an entity linking tool. Entity samples with confidence scores higher than a preset value are retained to construct training data that includes the input context and the target entity name; the input context and the target entity name correspond one-to-one.

3. The method according to claim 1, characterized in that, The process of training the short entity recall model in an autoregressive manner to learn the mapping relationship for generating target entity names based on the input context includes: For each of the training data, the input context in the training data is input into the encoder of the short entity recall model, and the encoder performs semantic encoding on the input context to output the context semantic features corresponding to the input context. The contextual semantic features are fed into the decoder of the short entity recall model. Based on the contextual semantic features, the decoder uses sequential autoregression to determine the predicted token sequence of the target entity name in units of tokens. Calculate the loss value between the predicted token sequence and the real token sequence corresponding to the target entity name in the training data; Based on the loss value, the network parameters of the short entity recall model are updated through backpropagation algorithm to optimize the learning effect of the short entity recall model on the mapping relationship between the input context and the target entity name.

4. The method according to claim 3, characterized in that, The training process also includes: Before training, all the completed training data is randomly divided into a training set and a validation set according to a preset ratio; the training set is used for iterative updates of model parameters, and the validation set is used to monitor the model training effect. During the training process of the short entity recall model, after each preset round of training set traversal learning is completed, parameter updates are paused and the model verification process is started. The input context in the validation set is sequentially input into the short entity recall model in the current training phase to obtain the corresponding predicted entity name generated by the short entity recall model; Calculate the matching accuracy between each predicted entity name and the corresponding real target entity name in the validation set, and calculate the F1 score of the validation set; If the F1 score of the validation set reaches the preset target threshold, the model training effect is deemed satisfactory, and training can be terminated. If the F1 score of the validation set does not reach the preset target threshold, the types of unmatched samples are analyzed, and after adjusting the training parameters, iterative learning of the training set is resumed until the validation metric is met.

5. A device for constructing a short entity recall model, characterized in that, The device includes: A building module is used to construct a short entity recall model based on a pre-trained language model, consisting of an encoder and a decoder architecture. The encoder is used to semantically encode the input context, and the decoder is used to generate entity names in an autoregressive manner. The parameter size of the short entity recall model is related to the size of the vocabulary. The training module is used to input the training data into the short entity recall model after acquiring the training data, and train the short entity recall model in an autoregressive manner to learn the mapping relationship for generating target entity names based on the input context; the training data includes the input context and the corresponding target entity names; The configuration module is used to configure a restricted decoding strategy so that the entity names generated during the inference of the short entity recall model belong to a preset candidate entity set, thereby obtaining a trained short entity recall model. The recall module is used to input the context to be retrieved into the trained short entity recall model, and generate the entity name corresponding to the context to be retrieved through the trained short entity recall model, thereby realizing short entity recall.

6. The apparatus according to claim 5, characterized in that, The device also includes an acquisition module; The acquisition module is used to extract metadata of entities and the context of the entities in real short text corpora from a preset knowledge base; wherein, the metadata includes the entity's standard name, definition description, and application scenario description; semantic role labeling technology is used to identify the semantic components of the entities labeled in the context, and redundant statements unrelated to the entities are filtered to ensure that the input context focuses on the key association information of the short entities; the semantic association confidence between the context and the entities is calculated through an entity linking tool, and entity samples with confidence scores higher than a preset value are retained to construct training data including the input context and the target entity name; the input context and the target entity name correspond one-to-one.

7. The apparatus according to claim 5, characterized in that, The training module is specifically used to input the input context in the training data into the encoder of the short entity recall model for each training data, and to perform semantic encoding on the input context through the encoder to output the context semantic features corresponding to the input context. The contextual semantic features are fed into the decoder of the short entity recall model. Based on the contextual semantic features, the decoder uses sequential autoregression to determine the predicted token sequence of the target entity name in units of tokens. The loss value between the predicted token sequence and the real token sequence corresponding to the target entity name in the training data is calculated. Based on the loss value, the network parameters of the short entity recall model are updated through backpropagation algorithm to optimize the learning effect of the short entity recall model on the mapping relationship between the input context and the target entity name.

8. The apparatus according to claim 7, characterized in that, The training module is further used to randomly divide all the constructed training data into a training set and a validation set according to a preset ratio before training. The training set is used for iterative updates of model parameters, and the validation set is used to monitor the model training effect. During the training process of the short entity recall model, after completing a preset number of training set traversal learning rounds, parameter updates are paused and the model validation process is started. The input context in the validation set is sequentially input into the short entity recall model in the current training stage to obtain the corresponding predicted entity names generated by the short entity recall model. The matching accuracy of each predicted entity name with the corresponding real target entity name in the validation set is calculated, and the F1 value of the validation set is calculated. If the F1 value of the validation set reaches a preset target threshold, the model training effect is determined to be satisfactory, and training can be terminated. If the F1 value of the validation set does not reach the preset target threshold, the type of unmatched samples is analyzed, and after adjusting the training parameters, the iteration learning of the training set is continued until the validation index is satisfactory.

9. A control device, characterized in that, It includes a processor and a memory, the memory being used to store programs, instructions, or code, and the processor being used to execute the programs, instructions, or code in the memory to complete the method for constructing a short entity recall model as described in any one of claims 1-4.

10. A computer-readable storage medium, characterized in that, The system contains a computer program that is loaded by a processor to execute the method for constructing a short entity recall model as described in any one of claims 1-4.

Images