Named entity sampling method and device based on diffusion model

Abstract

The invention provides a named entity sampling method and device based on a diffusion model, and the method comprises the steps: carrying out the marking and coding of named entities in training text data, and obtaining the boundary information corresponding to each named entity in the training text data; gradually adding noise in the boundary information to obtain target training text data, and training a preset diffusion model to obtain a target diffusion model; in response to the obtained target text data, determining the sampling time step length of the target diffusion model in the next time step based on sampling state information predicted by the target diffusion model for the target text data in the current time step and the historical time step; and performing iterative sampling on the named entities in the target text data according to the sampling time step length by using the target diffusion model to obtain named entity information corresponding to the target text data output by the target diffusion model. Through the method, the accuracy, flexibility and efficiency of named entity recognition are improved.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to a named entity sampling method and apparatus based on a diffusion model. Background Technology

[0002] Currently, generative NER is gradually replacing traditional sequence labeling methods and becoming a research hotspot in the Named Entity Recognition (NER) task. The DiffusionNER (Boundary Diffusion for Named Entity Recognition) method models the named entity recognition method as a diffusion inversion process of boundary prediction.

[0003] Among them, the advantages of the boundary prediction diffusion inversion named entity recognition method include structure independence, support for the generation of multiple entities or nested structures, and strong controllability. However, this method uses a fixed interval sampling strategy (e.g., DDIM) by default in the inference stage. This may cause undersampling problems in high error regions under the condition of limited sampling steps, resulting in uneven entity distribution in the text scene and the error may be rapidly amplified at key boundary positions, reducing the accuracy of named entity recognition. In addition, this method does not have dynamic adaptation characteristics and cannot allocate sampling resources according to the current state, which reduces the flexibility and efficiency of named entity recognition. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a named entity sampling method and apparatus based on a diffusion model. This method involves annotating and encoding training text data to obtain boundary information corresponding to each named entity, and progressively adding noise to the boundary information. The pre-set diffusion model is then trained using the noise-added and annotated training text data to obtain a target diffusion model. When the target text data for which named entity sampling is desired is obtained, the sampling time step of the target diffusion model is determined based on the sampling state information predicted by the target diffusion model for the target text data at the current time step and historical time steps. The named entities in the target text data are then iteratively sampled according to the sampling time step to obtain named entity information. This optimizes the performance of the diffusion model in the backsampling stage through a dynamic time step prediction mechanism based on historical state awareness, improving the accuracy of named entity recognition and thus enhancing the flexibility and efficiency of named entity recognition.

[0005] This application provides a named entity sampling method based on a diffusion model, the named entity sampling method including: The named entities in the preset training text data are labeled and encoded to obtain the boundary information corresponding to each named entity in the training text data; Noise is gradually added to the boundary information to obtain target training text data, and a preset diffusion model is trained using the target training text data to obtain the target diffusion model. In response to obtaining the target text data for which named entity sampling is desired, the sampling time step of the target diffusion model in the next time step is determined based on the sampling state information predicted by the target diffusion model for the target text data at the current time step and historical time steps respectively. The target diffusion model is used to iteratively sample the named entities in the target text data according to the sampling time step to obtain the named entity information corresponding to the target text data output by the target diffusion model.

[0006] Furthermore, the step of annotating and encoding named entities in the preset training text data to obtain boundary information corresponding to each named entity in the training text data includes: The preset training text data is input into the pre-trained language model, and the language model is used to encode each word segmentation unit in the training text data to obtain the context information corresponding to each word segmentation unit output by the language model. Based on the context information, the boundary corresponding to each named entity in the training text data is determined, and the boundary is encoded and labeled to obtain the boundary information corresponding to each named entity.

[0007] Furthermore, the stepwise addition of noise to the boundary information to obtain target training text data includes: During the training process of the pre-defined diffusion model, the signal preservation rate corresponding to each time step in the training process is determined by using the pre-defined noise parameter mapping relationship. The cumulative signal hold rate for each time step is determined by multiplying the cumulative signal hold rate for each time step by the cumulative signal hold rate for each time step prior to that time step. Based on the cumulative signal preservation rate, noise is gradually added to the boundary information corresponding to each named entity at each time step to obtain target training text data constructed from the training text data.

[0008] Furthermore, the step of training a preset diffusion model using the target training text data to obtain the target diffusion model includes: The target training text data is input into the diffusion model, and the diffusion model is used to predict noise in the target training text data to obtain the predicted noise term corresponding to each time step in the training process. Based on the predicted noise term, the diffusion model is used to backsample and predict the target training text data at each time step in the training process to obtain the training named entity information corresponding to each time step in the training process of the target training text data, so as to train the model to learn to restore the noisy named entity boundary to the mapping relationship corresponding to the original named entity boundary. Based on the predicted noise term and the training named entity information, the noise reconstruction error corresponding to each time step in the training process is determined, and it is determined whether the noise reconstruction error shows convergence. If the noise reconstruction error does not converge, then the training of the diffusion model continues at the next time step. If the noise reconstruction error shows convergence, the training of the diffusion model is completed to obtain the target diffusion model.

[0009] Furthermore, determining the sampling time step size of the target diffusion model in the next time step based on the sampling state information predicted by the target diffusion model for the target text data at the current time step and historical time steps includes: Based on the target text data, determine the sampling state information predicted by the target diffusion model at the current time step, and obtain the historical sampling state information predicted by the target diffusion model at historical time steps; The sampling state information and the historical sampling state information are input into a preset time step predictor to obtain the sampling time step of the target diffusion model in the next time step, which is output by the time step predictor at the current time step.

[0010] Furthermore, determining the sampling state information predicted by the target diffusion model at the current time step based on the target text data includes: The target text data is input into the target diffusion model, and the diffusion model is used to perform noise prediction on the target training text data to obtain the target prediction noise term predicted by the target diffusion model at the current time step. Based on the target prediction noise term, the target diffusion model is used to sample and predict the target text data at the current time step to obtain the boundary sampling information and context sampling information corresponding to each named entity in the target text data output by the target diffusion model at the current time step. Based on the target prediction noise term, the boundary sampling information, the context sampling information, and the current time step, the sampling state information predicted by the target diffusion model at the current time step is determined.

[0011] Furthermore, the step of iteratively sampling named entities in the target text data using the target diffusion model according to the sampling time step to obtain the named entity information corresponding to the target text data output by the target diffusion model includes: Based on the sampling time step predicted at each time step, the target diffusion model is used to iteratively sample the named entities in the target text data at multiple time steps to obtain the predicted named entity information corresponding to the target text data output by the target diffusion model at each time step. For each time step, determine whether the predicted sampling time step size for that time step is less than or equal to a first preset value or whether the number of time steps in the iteration reaches a second preset value; If the predicted sampling time step size at this time step is greater than a first preset value and / or the number of iterated time steps does not reach a second preset value, then the target diffusion model continues iterative sampling for the next time step. If the sampling time step predicted at that time step is less than or equal to a first preset value or the number of time steps in the iteration reaches a second preset value, then the predicted named entity information accumulated and output by the target diffusion model at that time step is determined as the named entity information corresponding to the target text data.

[0012] This application also provides a named entity sampling device based on a diffusion model, the named entity sampling device comprising: The annotation and encoding module is used to annotate and encode named entities in the preset training text data to obtain the boundary information corresponding to each named entity in the training text data. The model training module is used to gradually add noise to the boundary information to obtain target training text data, and use the target training text data to train a preset diffusion model to obtain the target diffusion model. The step size prediction module is used to determine the sampling time step of the target diffusion model in the next time step in response to the acquisition of target text data for which named entity sampling is expected, based on the sampling state information predicted by the target diffusion model for the target text data at the current time step and the historical time step respectively. The entity sampling module is used to iteratively sample named entities in the target text data according to the sampling time step using the target diffusion model, so as to obtain the named entity information corresponding to the target text data output by the target diffusion model.

[0013] Furthermore, when the annotation and encoding module is used to annotate and encode named entities in the preset training text data to obtain the boundary information corresponding to each named entity in the training text data, the annotation and encoding module is used to: The preset training text data is input into the pre-trained language model, and the language model is used to encode each word segmentation unit in the training text data to obtain the context information corresponding to each word segmentation unit output by the language model. Based on the context information, the boundary corresponding to each named entity in the training text data is determined, and the boundary is encoded and labeled to obtain the boundary information corresponding to each named entity.

[0014] Furthermore, when the model training module progressively adds noise to the boundary information to obtain the target training text data, the model training module is used to: During the training process of the pre-defined diffusion model, the signal preservation rate corresponding to each time step in the training process is determined by using the pre-defined noise parameter mapping relationship. The cumulative signal hold rate for each time step is determined by multiplying the cumulative signal hold rate for each time step by the cumulative signal hold rate for each time step prior to that time step. Based on the cumulative signal preservation rate, noise is gradually added to the boundary information corresponding to each named entity at each time step to obtain target training text data constructed from the training text data.

[0015] Furthermore, when the model training module trains a preset diffusion model using the target training text data to obtain the target diffusion model, the model training module is used to: The target training text data is input into the diffusion model, and the diffusion model is used to predict noise in the target training text data to obtain the predicted noise term corresponding to each time step in the training process. Based on the predicted noise term, the diffusion model is used to backsample and predict the target training text data at each time step in the training process to obtain the training named entity information corresponding to each time step in the training process of the target training text data, so as to train the model to learn to restore the noisy named entity boundary to the mapping relationship corresponding to the original named entity boundary. Based on the predicted noise term and the training named entity information, the noise reconstruction error corresponding to each time step in the training process is determined, and it is determined whether the noise reconstruction error shows convergence. If the noise reconstruction error does not converge, then the training of the diffusion model continues at the next time step. If the noise reconstruction error shows convergence, the training of the diffusion model is completed to obtain the target diffusion model.

[0016] Furthermore, when the step size prediction module determines the sampling time step of the target diffusion model in the next time step based on the sampling state information predicted by the target diffusion model for the target text data at the current time step and historical time steps, the step size prediction module is used to: Based on the target text data, determine the sampling state information predicted by the target diffusion model at the current time step, and obtain the historical sampling state information predicted by the target diffusion model at historical time steps; The sampling state information and the historical sampling state information are input into a preset time step predictor to obtain the sampling time step of the target diffusion model in the next time step, which is output by the time step predictor at the current time step.

[0017] Furthermore, when the step size prediction module is used to determine the sampling state information predicted by the target diffusion model at the current time step based on the target text data, the step size prediction module is used to: The target text data is input into the target diffusion model, and the diffusion model is used to perform noise prediction on the target training text data to obtain the target prediction noise term predicted by the target diffusion model at the current time step. Based on the target prediction noise term, the target diffusion model is used to sample and predict the target text data at the current time step to obtain the boundary sampling information and context sampling information corresponding to each named entity in the target text data output by the target diffusion model at the current time step. Based on the target prediction noise term, the boundary sampling information, the context sampling information, and the current time step, the sampling state information predicted by the target diffusion model at the current time step is determined.

[0018] Furthermore, when the entity sampling module iteratively samples named entities in the target text data according to the sampling time step using the target diffusion model to obtain named entity information corresponding to the target text data output by the target diffusion model, the entity sampling module is used to: Based on the sampling time step predicted at each time step, the target diffusion model is used to iteratively sample the named entities in the target text data at multiple time steps to obtain the predicted named entity information corresponding to the target text data output by the target diffusion model at each time step. For each time step, determine whether the predicted sampling time step size for that time step is less than or equal to a first preset value or whether the number of time steps in the iteration reaches a second preset value; If the predicted sampling time step size at this time step is greater than a first preset value and / or the number of iterated time steps does not reach a second preset value, then the target diffusion model continues iterative sampling for the next time step. If the sampling time step predicted at that time step is less than or equal to a first preset value or the number of time steps in the iteration reaches a second preset value, then the predicted named entity information accumulated and output by the target diffusion model at that time step is determined as the named entity information corresponding to the target text data.

[0019] This application embodiment also provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the name entity sampling method based on the diffusion model described above are performed.

[0020] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the name entity sampling method based on the diffusion model described above.

[0021] The naming entity sampling method and apparatus based on a diffusion model provided in this application include: annotating and encoding named entities in preset training text data to obtain boundary information corresponding to each named entity in the training text data; progressively adding noise to the boundary information to obtain target training text data, and using the target training text data to train a preset diffusion model to obtain a target diffusion model; in response to obtaining target text data for which named entity sampling is desired, determining the sampling time step of the target diffusion model in the next time step based on the sampling state information predicted by the target diffusion model for the target text data at the current time step and historical time steps; and iteratively sampling the named entities in the target text data using the target diffusion model according to the sampling time step to obtain the named entity information corresponding to the target text data output by the target diffusion model.

[0022] Compared to existing technologies that use a fixed-interval sampling strategy by default during the inference phase, this method obtains boundary information for each named entity by annotating and encoding the training text data. Noise is then progressively added to the boundary information. The pre-defined diffusion model is trained using the noisy and annotated training text data to obtain the target diffusion model. When the target text data for which named entities are to be sampled is obtained, the sampling time step of the target diffusion model is determined based on the sampling state information predicted by the target diffusion model for the target text data at the current time step and historical time steps. The named entities in the target text data are iteratively sampled according to the sampling time step to obtain the named entity information. This optimizes the performance of the diffusion model in the backsampling phase through a dynamic time step prediction mechanism based on historical state awareness, thereby improving the accuracy of named entity recognition and thus enhancing the flexibility and efficiency of named entity recognition.

[0023] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A flowchart illustrating a named entity sampling method based on a diffusion model provided in this application embodiment; Figure 2 A schematic diagram of a named entity sampling device based on a diffusion model provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this application.

[0027] Research has found that generative NER is gradually replacing traditional sequence labeling methods and has become a research hotspot in the Named Entity Recognition (NER) task. The DiffusionNER (Boundary Diffusion for Named Entity Recognition) method models the named entity recognition method as a diffusion inversion process of boundary prediction.

[0028] Among them, the advantages of the boundary prediction diffusion inversion named entity recognition method include structure independence, support for the generation of multiple entities or nested structures, and strong controllability. However, this method uses a fixed interval sampling strategy (e.g., DDIM) by default in the inference stage. This may cause undersampling problems in high error regions under the condition of limited sampling steps, resulting in uneven entity distribution in the text scene and the error may be rapidly amplified at key boundary positions, reducing the accuracy of named entity recognition. In addition, this method does not have dynamic adaptation characteristics and cannot allocate sampling resources according to the current state, which reduces the flexibility and efficiency of named entity recognition.

[0029] Based on this, this application provides a named entity sampling method based on a diffusion model. By annotating and encoding training text data to obtain boundary information corresponding to each named entity, noise is gradually added to the boundary information. The pre-set diffusion model is trained using the noise-added and annotated training text data to obtain a target diffusion model. When the target text data for which named entity sampling is desired is obtained, the sampling time step of the target diffusion model in the next time step is determined based on the sampling state information predicted by the target diffusion model for the target text data at the current time step and historical time steps. The named entities in the target text data are iteratively sampled according to the sampling time step to obtain named entity information. This optimizes the performance of the diffusion model in the backsampling stage through a dynamic time step prediction mechanism based on historical state awareness, improving the accuracy of named entity recognition and thus enhancing the flexibility and efficiency of named entity recognition.

[0030] Please see Figure 1 , Figure 1 This is a flowchart illustrating a named entity sampling method based on a diffusion model, provided as an embodiment of this application. Figure 1 As shown in the embodiments of this application, the named entity sampling method based on the diffusion model includes: S101. The named entities in the preset training text data are labeled and encoded to obtain the boundary information corresponding to each named entity in the training text data.

[0031] In this embodiment of the application, the preset training text data may include pre-set text data used to train the diffusion model.

[0032] Named Entity Recognition (NER) is a fundamental task in Natural Language Processing (NLP). It aims to extract semantically meaningful entities such as names of people, places, and organizations from natural language text. It is an important foundational task for information extraction and natural language understanding in NLP. Its goal is to identify entities with specific meanings from a text and classify them into predefined categories.

[0033] Furthermore, DiffusionNER (Boundary Diffusion for Named Entity Recognition) is a novel method that models the Named Entity Recognition (NER) task as a diffusion inversion process of boundary prediction. It draws on the idea of ​​diffusion models in the field of image generation and applies it to structured text tasks. Unlike traditional sequence labeling methods, DiffusionNER does not directly predict the label of each word. Instead, it gradually infers the start and end positions of entities, i.e., the boundary information of named entities, from randomly initialized boundary information through progressive denoising.

[0034] For example, in named entity recognition based on a diffusion model, each sentence is modeled as a set of potential entity boundaries. For instance, a text data sentence "Zhang San founded Company B in location A in 2000" might have the following boundaries: Entity 1 is [0, 2], corresponding to "Zhang San" (person); Entity 2 is [6, 11], corresponding to "2000" (time); Entity 3 is [14, 16], corresponding to "location A" (place); and Entity 4 is [19, 22], corresponding to "Company B" (organization).

[0035] In one embodiment of this application, step S101 may include: S1011. Input the preset training text data into the pre-trained language model, and use the language model to encode each word segmentation unit in the training text data to obtain the context information corresponding to each word segmentation unit output by the language model.

[0036] Here, the pre-trained language model may include, but is not limited to, a BERT (Bidirectional Encoder Representations from Transformers) model.

[0037] In this embodiment of the application, the context information corresponding to each word segmentation unit is represented as follows.

[0038] .

[0039] Among them, h i This represents the context information corresponding to each word segmentation unit; (x1,…,x n ) represents the preset training text data; i represents each word segmentation unit in the training text data; n represents the number of word segmentation units in the training text data.

[0040] S1012. Based on the context information, determine the boundary corresponding to each named entity in the training text data, and encode and label the boundary to obtain the boundary information corresponding to each named entity.

[0041] In this embodiment of the application, each named entity is represented as a boundary pair (s, e); where s and e are the start and end positions of the named entity in the training text data, respectively. The name entity recognition based on the diffusion model models the boundary of each named entity as a real number vector, as shown below.

[0042] .

[0043] Furthermore, the boundary information corresponding to each named entity is represented as follows.

[0044] .

[0045] in, This represents the boundary information corresponding to each named entity; Indicates the starting position corresponding to each named entity; Indicates the end position corresponding to each named entity; Represents a real number vector.

[0046] S102. Noise is gradually added to the boundary information to obtain target training text data, and the target training text data is used to train a preset diffusion model to obtain a target diffusion model.

[0047] In the embodiments of this application, training a preset diffusion model includes two steps: a forward diffusion process and a reverse denoising process, to achieve a diffusion inversion process for boundary prediction.

[0048] In the forward diffusion process, Gaussian noise is added to the real entity boundaries in the training text data in multiple steps, so that the entity boundary states of the obtained target training text data are close to the standard Gaussian distribution.

[0049] In the reverse process, a diffusion model is trained to learn the mapping relationship from the boundary information of each named entity in the training text data from the noisy state to the real state and to predict the denoising performance.

[0050] In one embodiment of this application, in specific implementation, the step S102 of progressively adding noise to the boundary information to obtain the target training text data may include: S1021. During the training process of the preset diffusion model, the signal retention rate corresponding to each time step in the training process is determined by using the preset noise parameter mapping relationship.

[0051] In this embodiment of the application, the expression for determining the signal retention rate corresponding to each time step in the training process is as follows.

[0052] .

[0053] in, This represents the signal hold rate at each time step; t This represents each time step in the training process; This indicates the preset noise parameter mapping relationship.

[0054] S1022. The signal hold rate corresponding to each time step is cumulatively multiplied by the signal hold rate of each time step prior to that time step to determine the cumulative signal hold rate corresponding to each time step.

[0055] In this step, the signal hold rate corresponding to each time step is cumulatively multiplied by the signal hold rate corresponding to each historical time step before that time step to obtain the product result, and the product result is determined as the cumulative signal hold rate corresponding to each time step.

[0056] S1023. Based on the cumulative signal preservation rate, noise is gradually added to the boundary information corresponding to each named entity at each time step to obtain target training text data constructed from the training text data.

[0057] In this embodiment, the formula for progressively adding noise to the boundary information corresponding to each named entity at each time step is shown below.

[0058] .

[0059] in, This represents the boundary information after adding noise to the boundary information corresponding to each named entity at each time step; This represents the initial boundary information corresponding to each named entity; This indicates the added noise; Indicates the cumulative signal hold rate; This indicates the preset noise range value.

[0060] In one embodiment of this application, in specific implementation, step S102, which involves training a preset diffusion model using the target training text data to obtain the target diffusion model, may include: S1024. Input the target training text data into the diffusion model, and use the diffusion model to perform noise prediction on the target training text data to obtain the predicted noise term corresponding to each time step in the training process.

[0061] In this embodiment of the application, the expression for noise prediction of the target training text data is as follows.

[0062] .

[0063] in, This represents the prediction noise term corresponding to each time step; Represents the noise prediction function; This represents the boundary information of the training text data used by the diffusion model to predict the target. This represents a three-dimensional tensor formed by fusing the contextual information and noise fragment information of each named entity in the target training text data. This represents each time step.

[0064] Here, the calculation process for the boundary information and the three-dimensional tensor in the noise prediction function is as follows.

[0065] Specifically, when calculating the three-dimensional tensor after fusing the context information and noise fragment information of each named entity in the target training text data, the dimension of the three-dimensional tensor is K×M×h, where K is the total number of expanded named entities and noise fragments, M represents the number of named entities, and h represents the dimension of the hidden layer.

[0066] Furthermore, the boundary information of the target training text data predicted by the diffusion model serves to provide a joint representation for subsequent boundary prediction and entity classification. In the process of generating this boundary information, a preset first learnable weight matrix is ​​used to perform a linear transformation on the context information of each named entity to adjust the representation space of the context information, and a preset second learnable weight matrix is ​​used to perform a linear transformation on the noise segment to adjust the representation space of the noise segment. The extended context information and the noise segment are added element by element to obtain the boundary information.

[0067] S1025. Based on the predicted noise term, the diffusion model is used to backsample and predict the target training text data at each time step in the training process to obtain the training named entity information corresponding to each time step in the training process of the target training text data, so as to train the model to learn to restore the noisy named entity boundary to the mapping relationship corresponding to the original named entity boundary.

[0068] In this step, the diffusion model performs backsampling prediction on the target training text data at each time step. First, it determines the cumulative signal preservation rate of the previous time step and the cumulative signal preservation rate of the current time step. Then, it combines the predicted noise term and the training named entity information of the previous time step to determine the training named entity information corresponding to the current time step of the target training text data in the training process. Finally, it obtains the training named entity information corresponding to each time step of the target training text data in the training process.

[0069] In this embodiment of the application, the formula for backsampling and predicting the target training text data at each time step of the diffusion model during the training process is shown below.

[0070] .

[0071] in, This represents the training named entity information corresponding to the current time step; This indicates the training named entity information for the previous time step, which is backsampled for prediction. Represents the predicted noise term; Indicates the preset noise parameters; This represents the cumulative signal hold-up rate of the previous time step predicted by backsampling; This indicates the cumulative signal hold rate at the current time step; This represents the context information corresponding to each named entity; This represents each time step.

[0072] S1026. Based on the predicted noise term and the training named entity information, determine the noise reconstruction error corresponding to each time step in the training process, and determine whether the noise reconstruction error shows convergence.

[0073] In this embodiment of the application, the expression for determining the noise reconstruction error corresponding to each time step is as follows.

[0074] .

[0075] in, This represents the noise reconstruction error corresponding to each time step; Represents the predicted noise term; Indicates the actual noise term; This represents the training named entity information corresponding to each time step; This represents the context information corresponding to each named entity; This represents each time step.

[0076] S1027. If the noise reconstruction error does not converge, then continue training the diffusion model at the next time step.

[0077] S1028. If the noise reconstruction error shows convergence, then the training of the diffusion model is completed to obtain the target diffusion model.

[0078] In this embodiment of the application, the goal of training the diffusion model is to minimize the noise reconstruction error. When the noise reconstruction error converges to a minimum, the training of the diffusion model is completed to obtain the target diffusion model; if the noise reconstruction error does not converge, the training of the diffusion model continues at the next time step.

[0079] S103. In response to obtaining the target text data for which named entity sampling is desired, the sampling time step of the target diffusion model in the next time step is determined based on the sampling state information predicted by the target diffusion model for the target text data at the current time step and the historical time step.

[0080] In the embodiments of this application, the fixed step size strategy has limitations when dealing with complex text with multiple nested entities. In order to schedule sampling more finely, a time step predictor is designed to adaptively determine the sampling time step size of the next time step based on the current sampling state of the target diffusion model.

[0081] Thus, the goal of determining the sampling time step of the target diffusion model in the next time step is to reduce the sampling time step in text regions where the sampling state shows a large error, and to increase the sampling time step in the stage where the accuracy of the sampling state is relatively stable.

[0082] Here, by dynamically determining the sampling time step size of the target diffusion model at each time step, the accumulation of errors during the sampling process can be reduced, the accuracy of generated entity boundaries can be improved, and the stability and robustness of the target diffusion model in the inference stage can be improved. At the same time, the sampling efficiency of named entities of the target diffusion model can be improved by optimizing the sampling strategy.

[0083] In one embodiment of this application, in specific implementation, step S103, which determines the sampling time step size of the target diffusion model in the next time step based on the sampling state information predicted by the target diffusion model for the target text data at the current time step and historical time steps respectively, may include: S1031. Based on the target text data, determine the sampling state information predicted by the target diffusion model at the current time step, and obtain the historical sampling state information predicted by the target diffusion model at historical time steps.

[0084] In one embodiment of this application, step S1031 may include: S10311. Input the target text data into the target diffusion model, and use the diffusion model to perform noise prediction on the target training text data to obtain the target prediction noise term predicted by the target diffusion model at the current time step.

[0085] In this embodiment of the application, the target diffusion model performs noise prediction on the target text data to obtain the target prediction noise item predicted at the current time step, and removes the predicted target prediction noise item during the sampling prediction process.

[0086] S10312. Based on the target prediction noise term, the target diffusion model is used to sample and predict the target text data at the current time step to obtain the boundary sampling information and context sampling information corresponding to each named entity in the target text data output by the target diffusion model at the current time step.

[0087] In this step, the target diffusion model removes target prediction noise from the target text data at the current time step in order to perform sampling prediction. At the current time step, the target diffusion model outputs the boundary sampling information and context sampling information corresponding to each named entity in the target text data.

[0088] S10313. Based on the target prediction noise term, the boundary sampling information, the context sampling information, and the current time step, determine the sampling state information predicted by the target diffusion model at the current time step.

[0089] In this embodiment of the application, the expression for determining the sampled state information predicted at the current time step is as follows.

[0090] .

[0091] in, This represents the sampled state information predicted at the current time step; This represents the target prediction noise term; Indicates boundary sampling information; This indicates contextual sampling information; Indicates the current time step.

[0092] S1032. Input the sampling state information and the historical sampling state information into a preset time step predictor to obtain the sampling time step of the target diffusion model in the next time step, which is output by the time step predictor at the current time step.

[0093] Here, the time step predictor may include a GRU model, a Transformer model, and an attention module, etc., and the time step predictor has the ability to model time series.

[0094] In this embodiment of the application, the expression for the sampling time step of the target diffusion model output by the time step predictor at the current time step in the next time step is as follows.

[0095] .

[0096] in, tnext This indicates the sampling time step size of the target diffusion model in the next time step; Indicates sampling status information; Indicates historical sampling status information; This represents the timing prediction function of the time-step predictor.

[0097] S104. The target diffusion model is used to iteratively sample the named entities in the target text data according to the sampling time step to obtain the named entity information corresponding to the target text data output by the target diffusion model.

[0098] In one embodiment of this application, step S104 may include: S1041. Based on the sampling time step predicted at each time step, the target diffusion model is used to iteratively sample the named entities in the target text data at multiple time steps to obtain the predicted named entity information corresponding to the target text data output by the target diffusion model at each time step.

[0099] Here, the predicted named entity information output at each time step is the sampling prediction result of the target diffusion model on the named entities in the target text data at each time step.

[0100] In this embodiment of the application, based on the sampling time step predicted at each time step, the noise removal parameters of the target diffusion model when iteratively sampling named entities in the target text data at multiple time steps can be determined. The expression of the noise removal parameters is as follows.

[0101] .

[0102] Wherein, represents the signal hold-up rate corresponding to each time step, i.e., the noise removal parameter; tnext Indicates the sampling time step size for the next time step; This indicates the preset noise parameter mapping relationship.

[0103] Furthermore, the expression for the denoising update performed by the target diffusion model when iteratively sampling named entities in the target text data is shown below.

[0104] .

[0105] in, This represents the predicted named entity information for the next time step predicted at each time step. This represents the prediction noise term for each time step. Represents the predicted named entity information for each time step; This indicates the sampling time step size at the current time step; tnext Indicates the sampling time step size for the next time step; The formula representing iterative sampling.

[0106] S1042. For each time step, determine whether the predicted sampling time step length for that time step is less than or equal to a first preset value or whether the number of time steps in the iteration reaches a second preset value.

[0107] In the embodiments of this application, the first preset value is generally set to 0; the second preset value can be determined according to the performance of the target diffusion model and the actual situation of the target text data.

[0108] S1043. If the predicted sampling time step size at this time step is greater than a first preset value and / or the number of iterated time steps does not reach a second preset value, then the target diffusion model continues iterative sampling for the next time step.

[0109] S1044. If the sampling time step predicted at this time step is less than or equal to a first preset value or the number of time steps in the iteration reaches a second preset value, then the predicted named entity information accumulated and output by the target diffusion model at this time step is determined as the named entity information corresponding to the target text data.

[0110] In this embodiment of the application, when the sampling time step is less than or equal to a first preset value (e.g., 0), or the number of iterated time steps reaches a second preset value, the predicted named entity information output by the target diffusion model at that time step and the predicted named entity information output before that time step are both determined as the named entity information corresponding to the target text data.

[0111] The named entity sampling method based on a diffusion model provided in this application obtains boundary information corresponding to each named entity by annotating and encoding training text data. Noise is gradually added to the boundary information. The pre-set diffusion model is trained using the training text data with added noise and annotation encoding to obtain a target diffusion model. When the target text data for which named entity sampling is desired is obtained, the sampling time step of the target diffusion model is determined based on the sampling state information predicted by the target diffusion model for the target text data at the current time step and the historical time step. The named entities in the target text data are iteratively sampled according to the sampling time step to obtain named entity information. This optimizes the performance of the diffusion model in the backsampling stage through a dynamic time step prediction mechanism based on historical state awareness, thereby improving the accuracy of named entity recognition and thus improving the flexibility and efficiency of named entity recognition.

[0112] Please see Figure 2 , Figure 2 This is a schematic diagram of a named entity sampling device based on a diffusion model, provided in an embodiment of this application. Figure 2 As shown, the named entity sampling device 200 includes: The annotation and encoding module 210 is used to annotate and encode named entities in the preset training text data to obtain boundary information corresponding to each named entity in the training text data. The model training module 220 is used to gradually add noise to the boundary information to obtain target training text data, and use the target training text data to train a preset diffusion model to obtain a target diffusion model. The step size prediction module 230 is used to determine the sampling time step of the target diffusion model in the next time step in response to obtaining the target text data for which named entity sampling is expected, based on the sampling state information predicted by the target diffusion model for the target text data at the current time step and the historical time step respectively. The entity sampling module 240 is used to iteratively sample the named entities in the target text data according to the sampling time step using the target diffusion model, so as to obtain the named entity information corresponding to the target text data output by the target diffusion model.

[0113] Furthermore, when the annotation and encoding module 210 is used to annotate and encode named entities in the preset training text data to obtain boundary information corresponding to each named entity in the training text data, the annotation and encoding module 210 is used to: The preset training text data is input into the pre-trained language model, and the language model is used to encode each word segmentation unit in the training text data to obtain the context information corresponding to each word segmentation unit output by the language model. Based on the context information, the boundary corresponding to each named entity in the training text data is determined, and the boundary is encoded and labeled to obtain the boundary information corresponding to each named entity.

[0114] Furthermore, when the model training module 220 progressively adds noise to the boundary information to obtain the target training text data, the model training module 220 is used to: During the training process of the pre-defined diffusion model, the signal preservation rate corresponding to each time step in the training process is determined by using the pre-defined noise parameter mapping relationship. The cumulative signal hold rate for each time step is determined by multiplying the cumulative signal hold rate for each time step by the cumulative signal hold rate for each time step prior to that time step. Based on the cumulative signal preservation rate, noise is gradually added to the boundary information corresponding to each named entity at each time step to obtain target training text data constructed from the training text data.

[0115] Furthermore, when the model training module 220 trains a preset diffusion model using the target training text data to obtain a target diffusion model, the model training module 220 is used to: The target training text data is input into the diffusion model, and the diffusion model is used to predict noise in the target training text data to obtain the predicted noise term corresponding to each time step in the training process. Based on the predicted noise term, the diffusion model is used to backsample and predict the target training text data at each time step in the training process to obtain the training named entity information corresponding to each time step in the training process of the target training text data, so as to train the model to learn to restore the noisy named entity boundary to the mapping relationship corresponding to the original named entity boundary. Based on the predicted noise term and the training named entity information, the noise reconstruction error corresponding to each time step in the training process is determined, and it is determined whether the noise reconstruction error shows convergence. If the noise reconstruction error does not converge, then the training of the diffusion model continues at the next time step. If the noise reconstruction error shows convergence, the training of the diffusion model is completed to obtain the target diffusion model.

[0116] Furthermore, when the step size prediction module 230 determines the sampling time step of the target diffusion model in the next time step based on the sampling state information predicted by the target diffusion model for the target text data at the current time step and historical time steps, the step size prediction module 230 is used to: Based on the target text data, determine the sampling state information predicted by the target diffusion model at the current time step, and obtain the historical sampling state information predicted by the target diffusion model at historical time steps; The sampling state information and the historical sampling state information are input into a preset time step predictor to obtain the sampling time step of the target diffusion model in the next time step, which is output by the time step predictor at the current time step.

[0117] Furthermore, when the step size prediction module 230 determines the sampling state information predicted by the target diffusion model at the current time step based on the target text data, the step size prediction module 230 is used to: The target text data is input into the target diffusion model, and the diffusion model is used to perform noise prediction on the target training text data to obtain the target prediction noise term predicted by the target diffusion model at the current time step. Based on the target prediction noise term, the target diffusion model is used to sample and predict the target text data at the current time step to obtain the boundary sampling information and context sampling information corresponding to each named entity in the target text data output by the target diffusion model at the current time step. Based on the target prediction noise term, the boundary sampling information, the context sampling information, and the current time step, the sampling state information predicted by the target diffusion model at the current time step is determined.

[0118] Furthermore, when the entity sampling module 240 iteratively samples the named entities in the target text data according to the sampling time step using the target diffusion model to obtain the named entity information corresponding to the target text data output by the target diffusion model, the entity sampling module 240 is used to: Based on the sampling time step predicted at each time step, the target diffusion model is used to iteratively sample the named entities in the target text data at multiple time steps to obtain the predicted named entity information corresponding to the target text data output by the target diffusion model at each time step. For each time step, determine whether the predicted sampling time step size for that time step is less than or equal to a first preset value or whether the number of time steps in the iteration reaches a second preset value; If the predicted sampling time step size at this time step is greater than a first preset value and / or the number of iterated time steps does not reach a second preset value, then the target diffusion model continues iterative sampling for the next time step. If the sampling time step predicted at that time step is less than or equal to a first preset value or the number of time steps in the iteration reaches a second preset value, then the predicted named entity information accumulated and output by the target diffusion model at that time step is determined as the named entity information corresponding to the target text data.

[0119] The named entity sampling device based on the diffusion model provided in this application obtains the boundary information corresponding to each named entity by annotating and encoding the training text data, and gradually adds noise to the boundary information. The preset diffusion model is trained using the training text data with added noise and annotation and encoding to obtain the target diffusion model. When the target text data for which named entity sampling is desired is obtained, the sampling time step of the target diffusion model is determined based on the sampling state information predicted by the target diffusion model for the target text data at the current time step and the historical time step, respectively. The named entities in the target text data are iteratively sampled according to the sampling time step to obtain the named entity information. This optimizes the performance of the diffusion model in the backsampling stage through a dynamic time step prediction mechanism based on historical state awareness, improves the accuracy of named entity recognition, and thus improves the flexibility and efficiency of named entity recognition.

[0120] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 3 As shown, the electronic device 300 includes a processor 310, a memory 320, and a bus 330.

[0121] The memory 320 stores machine-readable instructions executable by the processor 310. When the electronic device 300 is running, the processor 310 and the memory 320 communicate via the bus 330. When the machine-readable instructions are executed by the processor 310, they can perform the operations described above. Figure 1 The steps of the named entity sampling method based on the diffusion model in the method embodiment shown are specifically implemented in the method embodiment and will not be repeated here.

[0122] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figure 1 The steps of the named entity sampling method based on the diffusion model in the method embodiment shown are specifically implemented in the method embodiment and will not be repeated here.

[0123] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0124] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0125] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0126] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0127] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0128] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A named entity sampling method based on a diffusion model, characterized in that, The named entity sampling method includes: The named entities in the preset training text data are labeled and encoded to obtain the boundary information corresponding to each named entity in the training text data; Noise is gradually added to the boundary information to obtain target training text data, and a preset diffusion model is trained using the target training text data to obtain the target diffusion model. In response to obtaining the target text data for which named entity sampling is desired, the sampling time step of the target diffusion model in the next time step is determined based on the sampling state information predicted by the target diffusion model for the target text data at the current time step and historical time steps respectively. The target diffusion model is used to iteratively sample the named entities in the target text data according to the sampling time step to obtain the named entity information corresponding to the target text data output by the target diffusion model.

2. The method according to claim 1, characterized in that, The step of annotating and encoding named entities in the preset training text data to obtain boundary information corresponding to each named entity in the training text data includes: The preset training text data is input into the pre-trained language model, and the language model is used to encode each word segmentation unit in the training text data to obtain the context information corresponding to each word segmentation unit output by the language model. Based on the context information, the boundary corresponding to each named entity in the training text data is determined, and the boundary is encoded and labeled to obtain the boundary information corresponding to each named entity.

3. The method according to claim 1, characterized in that, The step of progressively adding noise to the boundary information to obtain target training text data includes: During the training process of the pre-defined diffusion model, the signal preservation rate corresponding to each time step in the training process is determined by using the pre-defined noise parameter mapping relationship. The cumulative signal hold rate for each time step is determined by multiplying the cumulative signal hold rate for each time step by the cumulative signal hold rate for each time step prior to that time step. Based on the cumulative signal preservation rate, noise is gradually added to the boundary information corresponding to each named entity at each time step to obtain target training text data constructed from the training text data.

4. The method according to claim 3, characterized in that, The step of training a preset diffusion model using the target training text data to obtain the target diffusion model includes: The target training text data is input into the diffusion model, and the diffusion model is used to predict noise in the target training text data to obtain the predicted noise term corresponding to each time step in the training process. Based on the predicted noise term, the diffusion model is used to backsample and predict the target training text data at each time step in the training process to obtain the training named entity information corresponding to each time step in the training process of the target training text data, so as to train the model to learn to restore the noisy named entity boundary to the mapping relationship corresponding to the original named entity boundary. Based on the predicted noise term and the training named entity information, the noise reconstruction error corresponding to each time step in the training process is determined, and it is determined whether the noise reconstruction error shows convergence. If the noise reconstruction error does not converge, then the training of the diffusion model continues at the next time step. If the noise reconstruction error shows convergence, the training of the diffusion model is completed to obtain the target diffusion model.

5. The method according to claim 1, characterized in that, The step of determining the sampling time step of the target diffusion model in the next time step based on the sampling state information predicted by the target diffusion model for the target text data at the current time step and historical time steps includes: Based on the target text data, determine the sampling state information predicted by the target diffusion model at the current time step, and obtain the historical sampling state information predicted by the target diffusion model at historical time steps; The sampling state information and the historical sampling state information are input into a preset time step predictor to obtain the sampling time step of the target diffusion model in the next time step, which is output by the time step predictor at the current time step.

6. The method according to claim 5, characterized in that, The step of determining the sampling state information predicted by the target diffusion model at the current time step based on the target text data includes: The target text data is input into the target diffusion model, and the diffusion model is used to perform noise prediction on the target training text data to obtain the target prediction noise term predicted by the target diffusion model at the current time step. Based on the target prediction noise term, the target diffusion model is used to sample and predict the target text data at the current time step to obtain the boundary sampling information and context sampling information corresponding to each named entity in the target text data output by the target diffusion model at the current time step. Based on the target prediction noise term, the boundary sampling information, the context sampling information, and the current time step, the sampling state information predicted by the target diffusion model at the current time step is determined.

7. The method according to claim 1, characterized in that, The step of iteratively sampling named entities in the target text data using the target diffusion model according to the sampling time step to obtain named entity information corresponding to the target text data output by the target diffusion model includes: Based on the sampling time step predicted at each time step, the target diffusion model is used to iteratively sample the named entities in the target text data at multiple time steps to obtain the predicted named entity information corresponding to the target text data output by the target diffusion model at each time step. For each time step, determine whether the predicted sampling time step size for that time step is less than or equal to a first preset value or whether the number of time steps in the iteration reaches a second preset value; If the predicted sampling time step size at this time step is greater than a first preset value and / or the number of iterated time steps does not reach a second preset value, then the target diffusion model continues iterative sampling for the next time step. If the sampling time step predicted at that time step is less than or equal to a first preset value or the number of time steps in the iteration reaches a second preset value, then the predicted named entity information accumulated and output by the target diffusion model at that time step is determined as the named entity information corresponding to the target text data.

8. A named entity sampling device based on a diffusion model, characterized in that, The named entity sampling device includes: The annotation and encoding module is used to annotate and encode named entities in the preset training text data to obtain the boundary information corresponding to each named entity in the training text data. The model training module is used to gradually add noise to the boundary information to obtain target training text data, and use the target training text data to train a preset diffusion model to obtain the target diffusion model. The step size prediction module is used to determine the sampling time step of the target diffusion model in the next time step in response to the acquisition of target text data for which named entity sampling is expected, based on the sampling state information predicted by the target diffusion model for the target text data at the current time step and the historical time step respectively. The entity sampling module is used to iteratively sample named entities in the target text data according to the sampling time step using the target diffusion model, so as to obtain the named entity information corresponding to the target text data output by the target diffusion model.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. The machine-readable instructions are executed by the processor to perform the steps of the named entity sampling method based on the diffusion model as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the named entity sampling method based on the diffusion model as described in any one of claims 1 to 7.

Images