Text processing method and device, equipment and readable storage medium

Abstract

The embodiment of the invention provides a text processing method and device, equipment and a readable storage medium. The method comprises the following steps: receiving a to-be-processed target text and a plurality of preset tags; the preset tag in the plurality of preset tags indicates one entity type. The method further comprises the steps of determining a plurality of intermediate recognition results corresponding to the plurality of preset tags based on the target text and the plurality of preset tags; the intermediate recognition result in the plurality of intermediate recognition results comprises the mention frequency of the mention word associated with the corresponding preset tag in the target text. And for at least a first preset tag in the plurality of preset tags, determining at least one first mention word associated with the first preset tag in the target text based on the target text and the intermediate recognition result corresponding to the first preset tag. In this way, the time for entity recognition can be significantly reduced, thereby reducing the delay in generating an entity recognition result.

Description

Technical Field

[0001] Example embodiments of the present disclosure generally relate to the field of computers, and more particularly, to methods, devices, apparatuses, and readable storage media for text processing. Background Art

[0002] Named Entity Recognition (NER) plays an important role in natural language processing. Its goal is to identify key information in text. In addition, NER also needs to classify these key information into predefined categories (also called tags), such as name, organization, place, time expression, quantity, monetary value, percentage, etc. By accurately identifying and classifying the key information in the text, it can provide the basis and support for subsequent natural language processing tasks. NER plays a vital role in knowledge base question answering, machine translation, information retrieval, text analysis and knowledge graph. Summary of the invention

[0003] In the first aspect of the present disclosure, a text processing method is provided. The method includes: receiving a target text to be processed and a plurality of preset tags, wherein a preset tag among the plurality of preset tags indicates an entity type; based on the target text and the plurality of preset tags, determining a plurality of intermediate recognition results corresponding to the plurality of preset tags respectively, wherein the intermediate recognition results among the plurality of intermediate recognition results include the number of times a mention word associated with the corresponding preset tag is mentioned in the target text; and for at least a first preset tag among the plurality of preset tags, determining at least one first mention word associated with the first preset tag in the target text based on the target text and the intermediate recognition results corresponding to the first preset tag.

[0004] In the second aspect of the present disclosure, a text processing device is provided. The device includes: a receiving module, configured to receive a target text to be processed and a plurality of preset tags, wherein a preset tag among the plurality of preset tags indicates an entity type; an intermediate recognition result determination module, configured to determine a plurality of intermediate recognition results corresponding to the plurality of preset tags respectively based on the target text and the plurality of preset tags, wherein the intermediate recognition results among the plurality of intermediate recognition results include the number of times a mention word associated with the corresponding preset tag is mentioned in the target text; and a mention word determination module, configured to determine, for at least a first preset tag among the plurality of preset tags, at least one first mention word associated with the first preset tag in the target text based on the target text and the intermediate recognition result corresponding to the first preset tag.

[0005] In a third aspect of the present disclosure, an electronic device is provided. The electronic device includes at least one processing unit; and at least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When the instructions are executed by the at least one processing unit, the electronic device executes the method of the first aspect.

[0006] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the method of the first aspect is implemented.

[0007] In a fifth aspect of the present disclosure, a computer program product is provided, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the method of the first aspect is implemented.

[0008] It should be understood that the content described in this section is not intended to limit the key features or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0010] Figure 1 A schematic diagram showing an example environment in which embodiments of the present disclosure can be implemented;

[0011] Figure 2 A schematic diagram showing a text processing architecture according to some embodiments of the present disclosure is shown;

[0012] Figure 3A A schematic diagram of constructing a first training sample according to some embodiments of the present disclosure is shown;

[0013] Figure 3B A schematic diagram of constructing a second training sample according to some embodiments of the present disclosure is shown;

[0014] Figure 4 A schematic diagram showing a text processing process according to some embodiments of the present disclosure;

[0015] Figure 5 A flowchart of a method for text processing according to some embodiments of the present disclosure is shown;

[0016] Figure 6 A schematic structural block diagram showing an apparatus for text processing according to some embodiments of the present disclosure; and

[0017] Figure 7 A block diagram of an electronic device in which one or more embodiments of the present disclosure may be implemented is shown. DETAILED DESCRIPTION

[0018] The embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0019] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0020] Herein, unless explicitly stated, executing a step “in response to A” does not mean executing the step immediately after “A” but may include one or more intermediate steps.

[0021] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition, use, storage or deletion of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0022] It is understandable that before using the technical solutions disclosed in each embodiment of the present disclosure, the types, scope of use, usage scenarios, etc. of the information involved in the present disclosure should be informed to relevant users and their authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations. The relevant users may include any type of right holders, such as individuals, enterprises, and groups.

[0023] For example, in response to receiving an active request from a user, a prompt message is sent to the relevant user to clearly prompt the relevant user that the operation requested to be performed will require obtaining and using the information of the relevant user, so that the relevant user can independently choose whether to provide information to software or hardware such as an electronic device, application, server or storage medium that executes the operation of the technical solution of the present disclosure based on the prompt message.

[0024] As an optional but non-limiting implementation, in response to receiving an active request from a relevant user, a prompt message is sent to the relevant user, for example, in the form of a pop-up window, in which the prompt message may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to select "agree" or "disagree" to provide information to the electronic device.

[0025] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0026] As used herein, the term "model" can learn the association between the corresponding input and output from the training data, so that after the training is completed, the corresponding output can be generated for a given input. The generation of the model can be based on machine learning technology. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs by using multi-layer processing units. A neural network model is an example of a model based on deep learning. In this article, "model" may also be referred to as "machine learning model", "learning model", "machine learning network" or "learning network", and these terms are used interchangeably in this article.

[0027] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. Environment 100 involves an electronic device 110, which can call a model 120 to perform a text processing task. Although shown as being deployed in electronic device 110, model 120 can be deployed in whole or in part on electronic device 110, or in whole or in part on a remote device. Model 120 can include a language model based on machine learning.

[0028] The electronic device 110 receives a text 102. The text 102 is, for example, an unstructured text. After identifying and classifying entities in the text 102, the electronic device 110 outputs an entity recognition result 104. The entity recognition result 104 is, for example, a text sequence including multiple elements.

[0029] The electronic device 110 can be any type of device with computing capabilities, including a terminal device or a server device. The terminal device can be any type of mobile terminal, fixed terminal or portable terminal, including a mobile phone, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a media computer, a multimedia tablet, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an e-book device, a gaming device, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. The server device can include, for example, a computing system / server, such as a mainframe, an edge computing node, a computing device in a cloud environment, and the like. In some embodiments, the management platform 110 can be implemented based on a cloud service.

[0030] It should be understood that the structure and functionality of environment 100 are described for exemplary purposes only and does not imply any limitation on the scope of the present disclosure.

[0031] As briefly mentioned above, the goal of NER is to identify key information in the text and classify it into predefined labels. For example, the text to be identified includes "On XX / XX / XXXX, AA Company announced the release of its latest product at its headquarters in BB City, which was an unexpected move." The sequence output based on the key information and classification can be expressed as "Date: XX / XX / XXXX, Company: AA Company, Location: BB City".

[0032] Current NER practices using language models are inefficient due to their inherent sequential processing, which autoregressively generates all label-key information pairs in a single sequence. This process significantly increases the sequence length, which increases the inference latency. In other words, the main bottleneck of language model inference latency is sequence length. The time complexity of language models is quadratic with respect to the length of the generated sequence, i.e., larger sequence lengths lead to longer inference latency.

[0033] In an embodiment of the present disclosure, an improved scheme for text processing is provided. The scheme can be implemented without additional modules or architectural changes. In the scheme, a target text to be processed and a plurality of preset tags are received. A preset tag among the plurality of preset tags indicates an entity type. Based on the target text and the plurality of preset tags, a plurality of intermediate recognition results corresponding to the plurality of preset tags are determined respectively. An intermediate recognition result among the plurality of intermediate recognition results includes the number of mentions of a mentioned word associated with the corresponding preset tag in the target text. For at least a first preset tag among the plurality of preset tags, at least one first mentioned word associated with the first preset tag in the target text is determined based on the target text and the intermediate recognition result corresponding to the first preset tag.

[0034] In this way, the time for entity recognition is reduced by allowing all tag mention pairs to be decoded with a shorter sequence length, thereby reducing the latency in generating entity recognition results. In addition, by decomposing entity recognition results from one sequence into multiple sequences, the difficulty of recognition can be reduced, thereby improving recognition quality.

[0035] Some example embodiments of the present disclosure will be described in detail below with reference to examples of the accompanying drawings.

[0036] Figure 2 Schematic diagram of a text processing architecture 200 according to some embodiments of the present disclosure is shown. The architecture 200 can be implemented in Figure 1 The architecture 200 is implemented in the environment 100, for example, at the electronic device 110. The architecture 200 involves a machine learning model 210 and a machine learning model 220. The machine learning model 210 and the machine learning model 220 can be considered as Figure 1 The following will refer to the example of model 120 in FIG. Figure 1 Give a description.

[0037] The electronic device 110 receives a target text 201 to be processed and a plurality of preset tags 202. Each preset tag 202 indicates an entity type. Examples of the preset tags 202 may include but are not limited to place, time, organization, and the like.

[0038] The electronic device 110 determines the intermediate recognition results 204 corresponding to each preset tag 202 based on the target text 201 and the plurality of preset tags 202. In some scenarios, depending on the content of the target text 201, part of the intermediate recognition results 204 may be empty, which means that the target text 201 does not contain the intermediate recognition results 204 corresponding to part of the preset tags 202.

[0039] The intermediate recognition result 204 includes the number of times the mentioned word associated with the corresponding preset tag 202 is mentioned in the target text 201. If the intermediate recognition result 204 corresponding to a preset tag 202 is empty, it means that the target text 201 does not contain the mentioned word associated with the preset tag 202.

[0040] In some embodiments, the electronic device 110 may determine the intermediate recognition result 204 using the machine learning model 210. Multiple instances of the machine learning model 210 may be deployed in the electronic device 110. The electronic device 110 may determine multiple intermediate recognition results 204 corresponding to multiple preset tags 202 respectively using a first number of instances of the machine learning model 210. The first number may be associated with the number of preset tags 202.

[0041] In some embodiments, the first number may be equal to the number of the plurality of preset tags 202. For example, 10 instances of the machine learning model 210 may be deployed in the electronic device 110, and the number of the preset tags 202 is 4, then the first number is 4. The electronic device 110 may use the 4 instances of the machine learning model 210 to determine 4 intermediate recognition results 204 corresponding to the 4 preset tags 202, respectively.

[0042] In some embodiments, the electronic device 110 may use the first number of instances of the machine learning model 210 to determine the intermediate recognition results 204 in parallel. In the above example, the electronic device 110 may use 4 instances in the machine learning model 210 to determine 4 intermediate recognition results 204 corresponding to 4 preset tags 202 in parallel.

[0043] In other embodiments, the first number may be less than the number of preset tags 202, and the electronic device 110 may use the first number of instances of the machine learning model 210 to partially parallelly determine the intermediate recognition result 204. For example, 10 instances of the machine learning model 210 may be deployed in the electronic device 110, the number of preset tags 202 is 4, and the first number is 2. The electronic device 110 may use 2 instances of the machine learning model 210. For example, the electronic device 110 may use one of the instances to determine the intermediate recognition results 204 corresponding to 2 preset tags 202 of the 4 preset tags 202, and use another instance to determine the intermediate recognition results 204 corresponding to the other 2 preset tags 202 of the 4 preset tags 202.

[0044] In some embodiments, the machine learning model 210 can be trained in the following manner: Taking the electronic device 110 performing the training process as an example, for one of the multiple preset tags 202, based on at least one mentioned word and the number of at least one mentioned word associated with the preset tag in the sample text, the electronic device 110 can construct a first training sample for the preset tag. Then, the electronic device 110 can train the machine learning model 210 based on the first training samples constructed for the multiple preset tags. It should be understood that the above-mentioned training of the machine learning model 210 can also be performed on other devices, and then deployed on the electronic device 110, or called by the electronic device 110, and the present disclosure does not limit this.

[0045] Figure 3A A schematic diagram of constructing a first training sample according to some embodiments of the present disclosure is shown. Sample text 302, for example, includes content "AAABBBCCCDDD". Preset tag 312 may indicate an entity type. When preset tag 312 is, for example, a person's name, it is expressed in the abbreviation form "PER", and its code format is "Entity type:\nPER\n\ <num>The preset tag 314 is, for example, miscellaneous, expressed in the abbreviated form "MIS", and its code format is "Entity type:\nMIS\n\ <num>The preset tag 316 is, for example, a location, expressed in the abbreviated form "LOC", and its code format is "Entitytype:\nLOC\n\ <num>The preset tag 318 is, for example, an organization, expressed in the abbreviation form "ORG", and its code format is "Entity type:\nORG\n\ <num>”.

[0046] The constructed first training sample includes the number of mentioned words and the mentioned words. For example, the first training sample 322 corresponding to the preset label 312 includes the number 1 and the mentioned word "AAA" associated with the name, the first training sample 324 corresponding to the preset label 314 includes the number 1 and the mentioned word "BBB" associated with the miscellaneous items, the first training sample 326 corresponding to the preset label 316 includes the number 2 and the mentioned words "CCC, DDD" associated with the place, and the first training sample 328 corresponding to the preset label 318 includes the number 0 and the mentioned words associated with the organization are empty.

[0047] use Figure 3A The method of constructing training samples shown in the figure can pair a preset label with one or more mentioned words. Such a construction method can be regarded as identifying mentioned words for preset labels.

[0048] Continue to refer Figure 2 , taking a preset tag 202 (also referred to as a first preset tag) among the plurality of preset tags 202 as an example, the electronic device 110 determines one or more mentioned words (also referred to as first mentioned words) associated with the preset tag 202 in the target text 201 based on the target text 201 and the intermediate recognition result 204 corresponding to the preset tag 202. The mentioned words may refer to nouns or phrases in the target text 201.

[0049] In some embodiments, the electronic device 110 may determine one or more mentioned words associated with the first preset tag based on the intermediate recognition result 204 corresponding to the first preset tag. In this way, a preset tag and a mentioned word pair may be formed. If the first number of mentions is 0, the first preset tag has no associated mentioned words.

[0050] Further, the electronic device 110 may determine whether the first mention count of one or more mentioned words in the target text 201 exceeds a predetermined count. For example, if the predetermined count is set to 1, the electronic device 110 may exclude preset tags with a mention count of 0, thereby only processing sequences that can form preset tag and mentioned word pairs in subsequent processing.

[0051] Furthermore, the electronic device 110 may determine the first mentioned word using the machine learning model 220 .

[0052] In some implementations, if the first number of mentions exceeds a predetermined number, the electronic device 110 may determine the first mention word using a second number of instances of the machine learning model 220. The second number may be associated with the first number of mentions.

[0053] In some embodiments, the second number may be equal to the sum of the first mentions. For example, the electronic device 110 may be deployed with 10 instances of the machine learning model 220, the number of preset tags 202 is 4, and the number of intermediate recognition results 204 is 4. Based on these intermediate recognition results 204, the electronic device 110 determines that the first mentions of the mentioned words associated with their corresponding preset tags 202 in the target text 201 are 1, 2, 1, and 1, respectively, then the second number may be 5. The electronic device 110 may use 5 instances of the machine learning model 220 to respectively determine the 5 mentioned words associated with the 4 preset tags 202, that is, each mention corresponds to one mentioned word.

[0054] In some embodiments, the electronic device 110 may determine the first mentioned word in parallel using the second number of instances of the machine learning model 220. In the above example, the electronic device 110 may determine the five mentioned words associated with the four preset tags 202 using five instances of the machine learning model 220 in parallel.

[0055] In other embodiments, the second number may be less than the sum of the first number of mentions, and the electronic device 110 may determine the first mention word partially in parallel using the second number of instances of the machine learning model 220. Continuing with the above example, the electronic device 110 may use 4 instances of the machine learning model 220, for example, first using the 4 instances to determine 4 mention words, and then using one of the instances to determine the 5th mention word.

[0056] In some embodiments, the machine learning model 210 and the machine learning model 220 may be the same model. For example, the electronic device 110 may call the same or different instances included in the model in different processing processes.

[0057] In some embodiments, the machine learning model 210 and the machine learning model 220 may be the same model (also referred to as a target machine learning model). The model may be trained in the following manner: Taking the electronic device 110 performing the training process as an example, for one preset tag 202 among the multiple preset tags 202, the electronic device 110 may construct at least one training sample corresponding to the at least one mentioned word based on at least one mentioned word associated with the preset tag 202 in the sample text, the number of the at least one mentioned word, and the mention identifier of each mentioned word in the at least one mentioned word. The mention identifier, for example, indicates the order number of the corresponding mentioned word.

[0058] Then, the electronic device 110 can train the model based on at least one training sample constructed for multiple preset tags. It should be understood that the above training of the model can also be performed on other devices, and then deployed on the electronic device 110, or called by the electronic device 110, and the present disclosure is not limited to this.

[0059] Figure 3B A schematic diagram of constructing a second training sample according to some embodiments of the present disclosure is shown. Sample text 302, for example, includes content "AAABBBCCCDDD". The description of the preset tag 312 can be referred to above and will not be repeated here. The constructed second training sample includes the number of mentioned words, the mention identifier and the mentioned words.

[0060] For example, the second training sample 332 corresponding to the preset label 312 includes the number 1, the mention word "AAA" associated with the name, and the mention identifier "mention 1" corresponding to the order in which the mention word appears; the second training sample 334 corresponding to the preset label 314 includes the number 1, the mention word "BBB" associated with the miscellaneous items, and the mention identifier "mention 1" corresponding to the order in which the mention word appears; the second training sample 336 corresponding to the preset label 316 includes the number 2, the mention word "CCC" associated with the place, and the mention identifier "mention 1" corresponding to the order in which the mention word appears; the second training sample 338 corresponding to the preset label 316 includes the number 2, the mention word "DDD" associated with the place, and the mention identifier "mention 2" corresponding to the order in which the mention word appears; the second training sample 340 corresponding to the preset label 318 includes the number 0 and the mention word associated with the organization is empty.

[0061] It can be seen from the second training sample 336 and the second training sample 338 that the mentioned words associated with the preset label 316 include "CCC" and "DDD", so the number of mentioned words is 2. The mentioned word "CCC" appears first in the sample text 302, so the mention identifier indicates that the order of occurrence of the mentioned word is 1. The mentioned word "DDD" appears later in the sample text 302, so the mention identifier indicates that the order of occurrence of the mentioned word is 2.

[0062] use Figure 3A The training samples are constructed in the manner shown, where a preset label may be paired with multiple mentioned words. Figure 3B By constructing training samples in the manner shown, preset labels can be paired with mentioned words one by one. In this way, training samples can be constructed at a smaller granularity, thereby improving the recognition efficiency and accuracy of the model.

[0063] Continue to refer Figure 2 , the electronic device 110 receives the intermediate recognition result 204, and uses the machine learning model 220 to determine the mentioned words associated with the preset tag 202 in the target text 201. Further, the electronic device 110 generates an entity recognition result 206 based on the determined mentioned words. The entity recognition result 206 is, for example, one or more sequences containing preset tags and their corresponding mentioned words. As an example, the target text 201 includes the content "AABB", and the preset tags include tag 1 and tag 2, then the entity recognition result 206 may include the sequence "tag 1: AA" and the sequence "tag 2: BB". The entity recognition result 206 may also be a combination of these sequences, such as the sequence "tag 1: AA, tag 2: BB".

[0064] References Figure 2 An architecture 200 for text processing is described and reference is made to Figure 3A and Figure 3B The construction of training samples for model training is described. By reformulating the training phase of the model, the embodiments of the present disclosure can use the trained model to predict the number of mentions of a specific tag and identify the mention words associated with the tag. Figure 4 Describes specific examples of using a trained machine learning model to process text.

[0065] Figure 4 FIG. 4 is a schematic diagram of a text processing process 400 according to some embodiments of the present disclosure. The process 400 may be implemented in Figure 1 The process 400 includes a first stage and a second stage. The electronic device 110 may use the machine learning model 210 in the first stage and the machine learning model 220 in the second stage. Figure 1 and Figure 2 Give a description.

[0066] The electronic device 110 receives a target text 201 and a plurality of preset tags. In the first stage, the electronic device 110 uses the machine learning model 210 to determine the number of times the mentioned words associated with the plurality of preset tags are mentioned in the target text 201, that is, the number of the mentioned words, such as Figure 4 As shown, they are 1, 2, 2 and 0 respectively.

[0067] In the second stage, the electronic device 110 uses the machine learning model 220 to determine the mention word corresponding to each mention, that is, the mention word corresponding to the mention identifier. Figure 4 As shown, the mention identifier 402 corresponds to the mention word 412, the mention identifier 404 corresponds to the mention word 414, the mention identifier 406 corresponds to the mention word 416, the mention identifier 408 corresponds to the mention word 418, and the mention identifier 410 corresponds to the mention word 420. Indirectly, these mention words and the corresponding preset tags constitute a preset tag and mention word pair.

[0068] In this way, the electronic device 110 can first predict the number of mentions of all preset tags, then predict all preset tags and mention word pairs in parallel, and finally summarize these preset tags and mention word pairs to obtain the entity recognition result.

[0069] like Figure 4 As shown, both the mentioned word 416 and the mentioned word 418 include "CC", however, "CC" only appears once in the target text 201. In this case, a deduplication operation is required to improve the recognition accuracy.

[0070] In some implementations, the electronic device 110 may, in response to determining that the same mentioned word is included in the plurality of mentioned words, perform a deduplication operation on the same mentioned word. Based on the result of the deduplication operation, the electronic device 110 determines an entity recognition result 206 for the target text 201 .

[0071] In some embodiments, the electronic device 110 can compare the probabilities of the same mentioned words being associated with the corresponding preset tags, and then remove the same mentioned words with lower probabilities. In this way, repeated mentions associated with preset tags can be eliminated based on the predicted probabilities. Figure 4 In the example of , the probability of mentioning word 416 is 0.61, and the probability of mentioning word 418 is 0.99, then the electronic device 110 can delete the mention of word 416. It should be understood that Figure 4 The probability values ​​of the mentioned words being associated with the preset tags are shown only for convenience of description and are not intended to be limiting.

[0072] In summary, the embodiments of the present disclosure can reconstruct the method of instruction tuning for parallel decoding of preset tags and mentioned word pairs, that is, first predict the number of mentions of the mentioned words associated with each preset tag in the target text, and then predict the specific mentioned words. In this way, the recognition time can be significantly reduced, thereby reducing the delay in generating entity recognition results. The embodiments of the present disclosure are not only applicable to NER tasks, but also to key information extraction applications that use any generative model to extract tag and mentioned word pairs. Furthermore, the embodiments of the present disclosure can aggregate the results of parallel decoding and use prediction probabilities to eliminate duplicate mentions, thereby improving recognition accuracy.

[0073] In addition, experiments show that the scheme of the present invention reduces the number of words (tokens) generated in each sequence in the NER task, reduces the inference delay by several to more than ten times, and improves the prediction quality to a certain extent compared with previous methods. The scheme of the present invention significantly reduces the inference time of the NER task by enabling parallel decoding, thereby solving the inefficiency problem of sequential decoding in LLM. While maintaining high accuracy and prediction quality comparable to or better than existing methods, the scheme of the present invention proves its effectiveness in flat and nested NER in English and Chinese.

[0074] Figure 5 1 is a flowchart of a method 500 for text processing according to some embodiments of the present disclosure. The method 500 may be implemented at the electronic device 110. Figure 1 Method 500 is described.

[0075] In block 510, the electronic device 110 receives a target text to be processed and a plurality of preset tags, wherein a preset tag among the plurality of preset tags indicates an entity type.

[0076] At block 520, the electronic device 110 determines a plurality of intermediate recognition results corresponding to the plurality of preset tags respectively based on the target text and the plurality of preset tags. The intermediate recognition results in the plurality of intermediate recognition results include the number of times the mention word associated with the corresponding preset tag is mentioned in the target text.

[0077] In box 530, for at least a first preset tag among the multiple preset tags, the electronic device 110 determines at least one first mentioned word associated with the first preset tag in the target text based on the target text and the intermediate recognition result corresponding to the first preset tag.

[0078] In some embodiments, the electronic device 110 uses a first number of instances of the first machine learning model to determine a plurality of intermediate recognition results corresponding to a plurality of preset tags, respectively, wherein the first number is associated with the number of the plurality of preset tags.

[0079] In some embodiments, the electronic device 110 determines whether the first number of mentions of a mention word associated with the first preset tag in the target text exceeds a predetermined number of times based on the intermediate recognition result corresponding to the first preset tag; and in response to the first number of mentions exceeding the predetermined number of times, determines at least one first mention word using a second number of instances of a second machine learning model, wherein the second number is associated with the first number of mentions.

[0080] In some embodiments, the first machine learning model is trained in the following manner: for a preset tag among multiple preset tags, based on at least one mentioned word associated with the preset tag and the number of at least one mentioned word in the sample text, constructing a first training sample for the preset tag; and training the first machine learning model based on the first training samples constructed respectively for multiple preset tags.

[0081] In some embodiments, generating multiple intermediate recognition results and determining at least one first mentioned word are achieved using a target machine learning model, and the target machine learning model is trained in the following manner: for a preset label among multiple preset labels, based on at least one mentioned word associated with the preset label in the sample text, the number of at least one mentioned word, and the mention identifier of each of the at least one mentioned word, at least one training sample corresponding to the at least one mentioned word is constructed; and based on at least one training sample constructed respectively for the multiple preset labels, the target machine learning model is trained.

[0082] In some embodiments, the electronic device 110 determines, for at least a second preset tag among multiple preset tags, based on the target text and the intermediate recognition result corresponding to the second preset tag, at least one second mentioned word associated with the second preset tag in the target text; and determines an entity recognition result for the target text based on at least one first mentioned word and at least one second mentioned word.

[0083] In some embodiments, the electronic device 110 performs a deduplication operation on the same mentioned word in response to determining that at least one first mentioned word and at least one second mentioned word include the same mentioned word; and determines an entity recognition result for the target text based on the result of the deduplication operation.

[0084] In some embodiments, the electronic device 110 compares a first probability that the same mentioned word is associated with a first preset tag and a second probability that the same mentioned word is associated with a second preset tag; and in response to the first probability being greater than the second probability, removes the same mentioned word from at least one second mentioned word.

[0085] Figure 6 A schematic structural block diagram of an apparatus 600 for text processing according to some embodiments of the present disclosure is shown. The apparatus 600 may be implemented in or included in the electronic device 110, for example. Each module / component in the apparatus 600 may be implemented by hardware, software, firmware, or any combination thereof.

[0086] As shown in the figure, the device 600 includes a receiving module 610, which is configured to receive a target text to be processed and a plurality of preset tags. A preset tag among the plurality of preset tags indicates an entity type. The device 600 also includes an intermediate recognition result determination module 620, which is configured to determine a plurality of intermediate recognition results corresponding to the plurality of preset tags respectively based on the target text and the plurality of preset tags. The intermediate recognition results among the plurality of intermediate recognition results include the number of mentions of the mentioned word associated with the corresponding preset tag in the target text. The device 600 also includes a mention word determination module 630, which is configured to determine, for at least a first preset tag among the plurality of preset tags, at least one first mention word associated with the first preset tag in the target text based on the target text and the intermediate recognition result corresponding to the first preset tag.

[0087] In some embodiments, the intermediate recognition result determination module 620 can be specifically configured to: use a first number of instances of the first machine learning model to determine multiple intermediate recognition results corresponding to multiple preset labels respectively, where the first number is associated with the number of multiple preset labels.

[0088] In some embodiments, the mention word determination module 630 can be specifically configured to: determine whether the first number of mentions of the mention word associated with the first preset label in the target text exceeds a predetermined number of times based on the intermediate recognition result corresponding to the first preset label; and in response to the first number of mentions exceeding the predetermined number of times, use a second number of instances of the second machine learning model to determine at least one first mention word, wherein the second number is associated with the first number of mentions.

[0089] In some embodiments, the first machine learning model is trained in the following manner: for a preset tag among multiple preset tags, based on at least one mentioned word associated with the preset tag and the number of at least one mentioned word in the sample text, constructing a first training sample for the preset tag; and training the first machine learning model based on the first training samples constructed respectively for multiple preset tags.

[0090] In some embodiments, generating multiple intermediate recognition results and determining at least one first mentioned word are achieved using a target machine learning model, and the target machine learning model is trained in the following manner: for a preset label among multiple preset labels, based on at least one mentioned word associated with the preset label in the sample text, the number of at least one mentioned word, and the mention identifier of each of the at least one mentioned word, at least one training sample corresponding to the at least one mentioned word is constructed; and based on at least one training sample constructed respectively for the multiple preset labels, the target machine learning model is trained.

[0091] In some embodiments, the device 600 also includes: a second mentioned word determination module, configured to determine, for at least a second preset tag among a plurality of preset tags, at least one second mentioned word associated with the second preset tag in the target text based on the target text and the intermediate recognition result corresponding to the second preset tag; and an entity recognition result determination module, configured to determine an entity recognition result for the target text based on at least one first mentioned word and at least one second mentioned word.

[0092] In some embodiments, the apparatus 600 further includes: a deduplication module configured to, in response to determining that at least one first mentioned word and at least one second mentioned word include the same mentioned word, perform a deduplication operation on the same mentioned word. The entity recognition result determination module may also be specifically configured to determine an entity recognition result for the target text based on the result of the deduplication operation.

[0093] In some embodiments, the device 600 also includes: a comparison module configured to compare a first probability of the same mentioned word being associated with a first preset tag and a second probability of the same mentioned word being associated with a second preset tag; and a removal module configured to remove the same mentioned word from at least one second mentioned word in response to the first probability being greater than the second probability.

[0094] Figure 7 1 shows a block diagram of an electronic device 700 in which one or more embodiments of the present disclosure may be implemented. It should be understood that Figure 7 The electronic device 700 shown is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein. Figure 7 The electronic device 700 shown may include or be implemented as Figure 1 Electronic device 110 or Figure 6 Device 600.

[0095] like Figure 7 As shown, the electronic device 700 is in the form of a general electronic device. The components of the electronic device 700 may include, but are not limited to, one or more processors or processing units 710, a memory 720, a storage device 730, one or more communication units 740, one or more input devices 750, and one or more output devices 760. The processing unit 710 may be an actual or virtual processor and is capable of performing various processes according to a program stored in the memory 720. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capability of the electronic device 700.

[0096] The electronic device 700 typically includes a plurality of computer storage media. Such media may be any accessible media that is accessible to the electronic device 700, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 720 may be a volatile memory (e.g., registers, caches, random access memory (RAM)), a non-volatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 730 may be a removable or non-removable medium, and may include a machine-readable medium, such as a flash drive, a disk, or any other medium, which may be capable of being used to store information and / or data and may be accessed within the electronic device 700.

[0097] The electronic device 700 may further include additional removable / non-removable, volatile / non-volatile storage media. Figure 7 As shown in , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. The memory 720 may include a computer program product 725 having one or more program modules that are configured to perform various methods or actions of various embodiments of the present disclosure.

[0098] The communication unit 740 implements communication with other electronic devices through a communication medium. Additionally, the functions of the components of the electronic device 700 can be implemented with a single computing cluster or multiple computing machines that can communicate through a communication connection. Therefore, the electronic device 700 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0099] The input device 750 may be one or more input devices, such as a mouse, a keyboard, a tracking ball, etc. The output device 760 may be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 700 may also communicate with one or more external devices (not shown) through the communication unit 740 as needed, such as a storage device, a display device, etc., communicate with one or more devices that allow a user to interact with the electronic device 700, or communicate with any device that allows the electronic device 700 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0100] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.

[0101] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices, equipment, and computer program products implemented according to the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0102] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0103] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0104] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0105] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The selection of terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the various implementations disclosed herein.< / num> < / num> < / num> < / num>

Claims

1. A text processing method, comprising: Receiving a target text to be processed and a plurality of preset tags, wherein a preset tag among the plurality of preset tags indicates an entity type; Based on the target text and the plurality of preset tags, determining a plurality of intermediate recognition results respectively corresponding to the plurality of preset tags, wherein an intermediate recognition result in the plurality of intermediate recognition results includes a number of times a mention word associated with the corresponding preset tag is mentioned in the target text; as well as For at least a first preset tag among the plurality of preset tags, based on the target text and the intermediate recognition result corresponding to the first preset tag, at least one first mentioned word associated with the first preset tag in the target text is determined.

2. The method according to claim 1, wherein determining the plurality of intermediate recognition results respectively corresponding to the plurality of preset tags comprises: Using a first number of instances of the first machine learning model, determine the plurality of intermediate recognition results corresponding to the plurality of preset tags respectively, wherein the first number is associated with the number of the plurality of preset tags.

3. The method according to claim 1, wherein determining the at least one first mentioned word associated with the first preset tag in the target text comprises: Based on the intermediate recognition result corresponding to the first preset tag, determining whether a first mention count of the mention word associated with the first preset tag in the target text exceeds a predetermined count; as well as In response to the first number of mentions exceeding a predetermined number, determining the at least one first mention word using a second number of instances of a second machine learning model, wherein the second number is associated with the first number of mentions.

4. The method according to claim 2, wherein the first machine learning model is trained by: For a preset tag among the plurality of preset tags, constructing a first training sample for the preset tag based on at least one mentioned word associated with the preset tag in the sample text and the number of the at least one mentioned word; and The first machine learning model is trained based on the first training samples respectively constructed for the multiple preset labels.

5. The method according to claim 1, wherein generating the plurality of intermediate recognition results and determining the at least one first mentioned word are implemented using a target machine learning model, and the target machine learning model is trained in the following manner: For a preset tag among the plurality of preset tags, based on at least one mentioned word associated with the preset tag in the sample text, the number of the at least one mentioned word, and a mention identifier of each of the at least one mentioned word, construct at least one training sample corresponding to the at least one mentioned word respectively; and The target machine learning model is trained based on the at least one training sample constructed respectively for the multiple preset labels.

6. The method according to claim 1, further comprising: For at least a second preset tag among the plurality of preset tags, determining, based on the target text and the intermediate recognition result corresponding to the second preset tag, at least one second mention word associated with the second preset tag in the target text; as well as Based on the at least one first mentioned word and the at least one second mentioned word, an entity recognition result for the target text is determined.

7. The method according to claim 6, wherein determining the entity recognition result for the target text comprises: In response to determining that the at least one first mentioned word and the at least one second mentioned word include the same mentioned word, performing a deduplication operation on the same mentioned word; as well as Based on the result of the deduplication operation, the entity recognition result for the target text is determined.

8. The method according to claim 7, wherein performing a deduplication operation on the same mentioned words comprises: Comparing a first probability of the same mentioned word being associated with the first preset tag and a second probability of the same mentioned word being associated with the second preset tag; as well as In response to the first probability being greater than the second probability, removing the same mentioned word from the at least one second mentioned word.

9. A device for text processing, comprising: A receiving module is configured to receive a target text to be processed and a plurality of preset tags, wherein a preset tag in the plurality of preset tags indicates an entity type; an intermediate recognition result determination module, configured to determine, based on the target text and the plurality of preset tags, a plurality of intermediate recognition results corresponding to the plurality of preset tags respectively, wherein an intermediate recognition result in the plurality of intermediate recognition results includes a number of times a mention word associated with the corresponding preset tag is mentioned in the target text; as well as The mentioned word determination module is configured to determine, for at least a first preset tag among the multiple preset tags, at least one first mentioned word associated with the first preset tag in the target text based on the target text and the intermediate recognition result corresponding to the first preset tag.

10. An electronic device, comprising: at least one processing unit; as well as At least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the electronic device to perform the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, the computer program being executable by a processor to implement the method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images