Business entity recognition method, device, equipment and storage medium

Abstract

The application discloses a business entity identification method and device, equipment and a storage medium, relates to the technical field of artificial intelligence, and comprises the following steps: performing data enhancement processing on a business watermark text to obtain an enhanced business text; performing special label annotation processing on the enhanced business text to obtain enhanced annotated text data; performing hierarchical fine-tuning training on a pre-training language model based on a full-word mask according to the enhanced annotated text data to obtain an initial business entity identification model; performing adversarial training on the initial business entity identification model through an adversarial interference generation strategy based on a model gradient to obtain a target business entity identification model; and inputting a target business text to be identified into the target business entity identification model for entity identification processing to obtain a business entity identification result, so that the business entity boundary identification precision of the business watermark text after optical character recognition, the ability to distinguish easily confused business entities, and the robustness of the model to noise interference are improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to methods, apparatus, devices and storage media for business entity identification. Background Technology

[0002] In highly compliant business scenarios such as finance and government affairs, business entity recognition is a key link in achieving automated processes, risk control, and compliance review. With the widespread application of optical character recognition technology in the digitization of business documents, accurate entity recognition of unstructured business text after optical character recognition has become an urgent industry need.

[0003] Currently, business entity recognition mainly relies on fine-tuning methods based on pre-trained language models. However, existing methods have limited ability to identify the boundaries of business entities, poor ability to distinguish easily confused entities that are semantically similar but have different business meanings, and insufficient adaptability to interference such as noise, typos, and blurred boundaries in the text after optical character recognition. They are difficult to meet the requirements of high-precision business scenarios.

[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main objective of this application is to provide a business entity identification method, apparatus, device, and storage medium, aiming to solve the technical problem of how to perform business entity identification on business watermark text after optical character recognition.

[0006] To achieve the above objectives, this application proposes a business entity identification method, which includes: Obtain the business watermark text after optical character recognition, and perform data enhancement processing on the business watermark text to obtain enhanced business text; The enhanced business text is labeled with special tags to obtain enhanced labeled text data; Based on the enhanced annotated text data, the pre-trained language model based on full-word masking is subjected to hierarchical fine-tuning training to obtain the initial business entity recognition model; The initial business entity identification model is adversarially trained using an adversarial perturbation generation strategy based on model gradients to obtain the target business entity identification model. The target business text to be identified is input into the target business entity recognition model for entity recognition processing to obtain the business entity recognition result.

[0007] In one embodiment, the step of obtaining the business watermark text after optical character recognition and performing data enhancement processing on the business watermark text to obtain enhanced business text includes: Obtain the business watermark text after optical character recognition; The watermarked text is divided into a training set and a verification set according to a preset ratio; The non-core entities in the training set are subjected to synonym replacement processing to obtain the first replacement text; The entities in the first replacement text are replaced to generate a virtual card number identifier and a virtual identity identifier, thus obtaining the second replacement text; The second replacement text is processed by initial and intermediate non-entity annotation to obtain the enhanced business text.

[0008] In one embodiment, the step of performing special tagging processing on the enhanced business text to obtain enhanced tagged text data includes: Identify easily confused entity pairs in the enhanced business text, wherein the easily confused entity pairs include a first easily confused entity and a second easily confused entity; Add a first special tag to the first easily confused entity; Add a second special tag to the second easily confused entity; Based on the first special label and the second special label, the first easily confused entity and the second easily confused entity are marked with boundary distinction to obtain enhanced labeled text data.

[0009] In one embodiment, the step of performing hierarchical fine-tuning training on a pre-trained language model based on whole-word masking according to the enhanced annotated text data to obtain an initial business entity recognition model includes: Obtain a pre-trained language model based on full-word masking; Determine the total number of hidden layers, the initial learning rate of the top layer, and the preset learning rate decay coefficient of the pre-trained language model; Based on the initial learning rate of the top layer and the preset learning rate decay coefficient, the learning rate of each hidden layer is calculated layer by layer downwards to obtain the target learning rate of each layer; The learning rate of the word embedding layer is calculated based on the total number of hidden layers and the preset learning rate decay coefficient, and the target learning rate of the word embedding layer is obtained. Based on the target learning rate of each layer and the target learning rate of the word embedding layer, the parameters of each layer of the pre-trained language model are updated differentially to obtain the pre-trained language model with updated parameters. The enhanced labeled text data is input into the pre-trained language model with updated parameters to train business entity recognition, thereby obtaining the initial business entity recognition model.

[0010] In one embodiment, the step of performing adversarial training on the initial business entity recognition model using an adversarial perturbation generation strategy based on model gradients to obtain the target business entity recognition model includes: The current training sample is input into the initial business entity recognition model for forward calculation to obtain the predicted label; Based on the predicted label and the true label of the current training sample, the loss function value is calculated using a joint loss function; Based on the loss function value, calculate the gradient values ​​of each parameter of the initial business entity identification model; Based on the gradient value and the preset perturbation coefficient, generate adversarial perturbation samples; The adversarial perturbation sample is input into the initial business entity recognition model for training to obtain the target business entity recognition model.

[0011] In one embodiment, the step of calculating the loss function value based on the predicted label and the true label of the current training sample using a joint loss function includes: Based on the entity boundary information of each word in the predicted label and the real label, the boundary constraint loss value is calculated using the boundary constraint loss function; Based on the predicted label and the preset smoothing coefficient, the predicted label is smoothed at the word level to obtain the smoothed predicted label. Calculate the smoothing loss value based on the smoothed predicted label and the true label; The loss function value is obtained by weighted summation of the boundary constraint loss value and the smoothing loss value.

[0012] In one embodiment, the step of inputting the target business text to be identified into the target business entity recognition model for entity recognition processing to obtain the business entity recognition result includes: Obtain the target business text to be identified; The target business text is input into the target business entity recognition model to obtain the entity label prediction probability of each word. Based on the entity label prediction probability and boundary constraints of each word, the entity category and boundary position of each word are determined. Based on the entity category of each word and the boundary position, a business entity identification result is generated.

[0013] Furthermore, to achieve the above objectives, this application also proposes a business entity identification device, the device comprising: The data augmentation annotation module is used to acquire the business watermark text after optical character recognition, and to perform data augmentation processing on the business watermark text to obtain the enhanced business text. The special labeling module is used to perform special labeling processing on the enhanced business text to obtain enhanced labeled text data; The hierarchical fine-tuning module is used to perform hierarchical fine-tuning training on the pre-trained language model based on the enhanced labeled text data to obtain the initial business entity recognition model. The adversarial training module is used to perform adversarial training on the initial business entity recognition model through an adversarial perturbation generation strategy based on model gradients, so as to obtain the target business entity recognition model. The entity recognition module is used to input the target business text to be recognized into the target business entity recognition model for entity recognition processing, and obtain the business entity recognition result.

[0014] In addition, to achieve the above objectives, this application also proposes a business entity identification device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the business entity identification method as described above.

[0015] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the business entity identification method described above.

[0016] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the business entity identification method described above.

[0017] One or more technical solutions proposed in this application have at least the following technical effects: By combining data augmentation with special labeling, the training samples can be expanded and the boundary differentiation information of easily confused entities can be strengthened. A layer-by-layer decreasing learning rate strategy is used to fine-tune the pre-trained language model, which can quickly adapt the top-level task parameters to the business entity recognition task while preserving the underlying general semantic knowledge, effectively improving the model's generalization ability in data-sparse scenarios. Adversarial training using a gradient-based adversarial perturbation generation method enables the model to actively learn the noise features and difficult sample distribution in the text after optical character recognition, significantly enhancing its robustness against noise, typos, and blurred boundary interference. In summary, these methods achieve high-precision boundary recognition and accurate category differentiation of business entities in the watermarked text after optical character recognition. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating an embodiment of the business entity identification method of this application. Figure 2 This is a flowchart illustrating the second embodiment of the business entity identification method in this application. Figure 3 This is a schematic diagram of the module structure of the business entity identification device according to an embodiment of this application; Figure 4 This is a schematic diagram of the device structure of the hardware operating environment involved in the business entity identification method in this application embodiment.

[0021] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0022] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0023] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0024] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or business entity identification device capable of performing the above functions. The following description uses a business entity identification device as an example to illustrate this embodiment and the subsequent embodiments.

[0025] Based on this, embodiments of this application provide a method for identifying business entities, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the business entity identification device method of this application.

[0026] In this embodiment, the method for identifying business entities includes steps S10 to S50: Step S10: Obtain the business watermark text after optical character recognition, and perform data enhancement processing on the business watermark text to obtain the enhanced business text; It should be noted that Optical Character Recognition (OCR) refers to the technology of scanning and recognizing text in paper documents or images using electronic devices, converting it into computer-editable text. Business watermarked text refers to text data with business-specific watermark information obtained after OCR processing; that is, text content extracted from paper documents or images in financial or government scenarios. Data augmentation refers to the process of increasing the diversity of training samples by performing word substitution and transformation on the original text without changing its core semantics. Enhanced business text refers to business text data obtained after data augmentation processing for subsequent model training.

[0027] Understandably, step S10 refers to acquiring the business watermark text after optical character recognition and performing data augmentation processing on the business watermark text to obtain enhanced business text. This step, by expanding and transforming the original business watermark text, provides richer and more diverse sample data for subsequent model training.

[0028] The beneficial effect of this step is that by performing data augmentation on the business watermark text, the scale of the training sample can be effectively expanded, the model's adaptability to diverse business texts can be improved, and a data foundation can be laid for subsequent high-precision entity recognition.

[0029] In the specific implementation, the business watermark text after OCR recognition is collected and divided into training set and validation set in a 5:5 ratio (6000 texts in total). Data augmentation is performed using synonym replacement (for non-core entities) and entity replacement (to generate virtual card numbers and IDs) to improve generalization capabilities; Introduce the BIO labeling scheme to add special labels to easily confused entities (such as "account manager's name") and clarify the boundary positions.

[0030] In one feasible implementation, step S10 may include steps A11 to A15: Step A11: Obtain the business watermark text after optical character recognition; Understandably, step A11 refers to acquiring the business watermark text after optical character recognition (OCR), thereby providing the raw data foundation for subsequent data augmentation and annotation processing. In this embodiment, the business watermark text originates from the OCR results of financial business documents.

[0031] The benefit of this step is that by acquiring the business watermark text after optical character recognition, the digitized business document content can be directly used to provide raw input for subsequent text processing.

[0032] Step A12: Divide the business watermark text into a training set and a verification set according to a preset ratio; It should be noted that the preset ratio refers to a pre-defined proportional parameter used to divide the dataset, specifically the ratio used in this embodiment to divide the business watermark text into a training set and a validation set. The training set refers to the set of sample data used to train the machine learning model, where the model learns text features and entity annotation patterns. The validation set refers to the set of sample data used to verify the model's training effectiveness and fine-tune parameters, i.e., the data set used to evaluate the model's generalization ability during training.

[0033] Understandably, step A12 refers to dividing the business watermark text into a training set and a validation set according to a preset ratio, so as to be used for model training and model validation respectively. In this embodiment, the preset ratio is five to five.

[0034] The beneficial effect of this step is that by dividing the business watermark text into training and validation sets according to a preset ratio, it can effectively monitor the generalization performance of the model during the training process while ensuring sufficient training samples, and prevent the model from overfitting.

[0035] Step A13: Perform synonym replacement on non-core entities in the training set to obtain the first replacement text; It should be noted that non-core entities refer to entity information in the business text that is not a key identification object, i.e., auxiliary entity content with little impact on the business entity identification task objective. Synonym replacement processing refers to replacing specified words in the text with semantically similar words to increase the diversity of textual expression while maintaining semantic consistency. The first replaced text refers to the intermediate text data obtained after synonym replacement processing.

[0036] Understandably, step A13 refers to performing synonym replacement processing on non-core entities in the training set to obtain the first replacement text, thereby achieving diversified expression of non-core semantic information.

[0037] The beneficial effect of this step is that by performing synonym replacement on non-core entities in the training set, the vocabulary diversity of the training samples can be increased while keeping the core semantics of the business text unchanged, thereby improving the model's ability to generalize and recognize synonymous expressions.

[0038] Step A14: Perform entity replacement processing on the entities in the first replacement text to generate virtual card number identifier and virtual identity identifier, thus obtaining the second replacement text; It should be noted that entity replacement processing refers to replacing real entity information in text with generated virtual information, a method that expands the training samples while protecting privacy. Virtual card number identifiers refer to virtual identifier information generated through entity replacement processing to replace real bank card numbers. Virtual identity identifiers refer to virtual identifier information generated through entity replacement processing to replace real identity information. Second replacement text refers to the intermediate text data obtained after entity replacement processing.

[0039] Understandably, step A14 refers to performing entity replacement processing on the entities in the first replacement text to generate virtual card number identifiers and virtual identity identifiers, thereby obtaining the second replacement text, achieving privacy protection of core entity information and sample expansion.

[0040] The beneficial effect of this step is that by performing entity replacement processing on the entities in the first replacement text to generate virtual card number identifiers and virtual identity identifiers, it is possible to expand the diversity of entity representations in the training samples while protecting customer privacy information, and enhance the robustness of the model in recognizing various card numbers and identity identifiers.

[0041] Step A15: Perform initial and intermediate non-entity annotation processing on the second replacement text to obtain the enhanced business text.

[0042] It should be noted that the start, middle and non-entity tagging processing refers to a sequence tagging method that uses start tags to mark the beginning position of entities, middle tags to mark the internal position of entities, and non-entity tags to mark the position of non-entities. That is, each word is assigned one of the three types of tags: start, middle and non-entity.

[0043] Understandably, step A15 refers to performing initial intermediate non-entity annotation processing on the second replacement text to obtain enhanced business text, providing labeled data with boundary information for model training.

[0044] The beneficial effect of this step is that by performing start and middle non-entity annotation processing on the second replacement text, it can provide clear entity boundary position information for subsequent model training, enabling the model to accurately learn the discrimination rules of entity start and end positions.

[0045] Step S20: Perform special tagging processing on the enhanced business text to obtain enhanced tagged text data; It should be noted that special label annotation processing refers to the annotation method of adding distinguishing markers to easily confused entities. That is, it enhances the model's ability to distinguish between semantically similar entities but with different business meanings by adding extra labels. Enhanced labeled text data refers to the final training data obtained after special label annotation processing, that is, text data that simultaneously contains boundary labeling information and easily confused entity distinguishing labels.

[0046] Understandably, step S20 refers to performing special labeling processing on the enhanced business text to obtain enhanced labeled text data. This step strengthens the model's ability to distinguish similar entities by adding special labels to easily confused entities. In this embodiment, the easily confused entities include entity pairs such as customer names and account manager names, which are semantically similar but have different business meanings.

[0047] The beneficial effect of this step is that by performing special labeling on the enhanced business text, we obtain enhanced labeled text data, which can effectively improve the model's ability to distinguish the boundaries of easily confused entities and reduce the recognition errors of similar entities such as customer names and account manager names.

[0048] In one feasible implementation, step S20 may include steps A21 to A25: Step A21: Identify easily confused entity pairs in the enhanced business text, including a first easily confused entity and a second easily confused entity; It should be noted that easily confused entity pairs refer to combinations of entities that have similar semantic expressions in business text but differ in business meaning; that is, similar entity combinations that are difficult for the model to directly distinguish. The first easily confused entity refers to the first entity member in an easily confused entity pair. The second easily confused entity refers to the second entity member in an easily confused entity pair.

[0049] Understandably, step A21 refers to identifying easily confused entity pairs in the enhanced business text, thereby determining the entity combinations that need to be specially distinguished.

[0050] The benefit of this step is that by identifying easily confused entity pairs in the enhanced business text, it is possible to accurately locate similar entity combinations that are difficult for the model to distinguish, providing a clear target for subsequently adding special labels.

[0051] Step A22: Add a first special tag to the first easily confused entity; It should be noted that the first special label refers to the special marking information added to distinguish the first easily confused entity, that is, the special symbol used to clearly identify the boundary and category of the first easily confused entity.

[0052] Understandably, step A22 refers to adding a first special label to the first easily confused entity, thereby giving the first easily confused entity distinguishable identification information.

[0053] The beneficial effect of this step is that by adding a first special label to the first easily confused entity, an independent label identifier can be established for the first easily confused entity, enabling the model to clearly distinguish the unique features of the first easily confused entity during the training process.

[0054] Step A23: Add a second special tag to the second easily confused entity; It should be noted that the second special label refers to the special marking information added to distinguish the second easily confused entity, that is, the special symbol used to clearly identify the boundary and category of the second easily confused entity.

[0055] Understandably, step A23 refers to adding a second special label to the second easily confused entity, thereby giving the second easily confused entity distinguishable identification information.

[0056] The beneficial effect of this step is that by adding a second special label to the second easily confused entity, an independent label identifier can be established for the second easily confused entity, so that the model can clearly distinguish the unique features of the second easily confused entity during the training process.

[0057] Step A24: Based on the first special label and the second special label, perform boundary differentiation annotation on the first easily confused entity and the second easily confused entity to obtain enhanced annotated text data.

[0058] Understandably, step A24 refers to performing boundary differentiation annotations on the first and second easily confused entities based on the first and second special tags, thereby obtaining enhanced annotated text data. This step achieves accurate boundary differentiation of easily confused entity pairs by jointly using the first and second special tags.

[0059] The beneficial effect of this step is that by labeling the boundaries of the first and second easily confused entities according to the first and second special labels, enhanced labeled text data is obtained. This enables the model to clearly learn the boundary differences between easily confused entities during the training process, significantly improving the accuracy of distinguishing similar entities.

[0060] Step S30: Based on the enhanced annotated text data, perform hierarchical fine-tuning training on the pre-trained language model based on full-word masking to obtain the initial business entity recognition model; It's important to note that Whole Word Masking (WWM) refers to a technique used during the training of a pre-trained language model to mask entire words, rather than individual characters. A pre-trained language model is a language understanding model pre-trained on a large-scale general corpus; it's a basic neural network model with general semantic understanding capabilities. Layered fine-tuning training refers to a training method that uses differentiated learning rates to adjust parameters of different network layers in the pre-trained language model. The lower layers use a smaller learning rate to retain general knowledge, while the upper layers use a larger learning rate to adapt to specific tasks. The initial business entity recognition model refers to the initial model obtained after layered fine-tuning training, used to identify entities in business texts.

[0061] Understandably, step S30 refers to performing hierarchical fine-tuning training on the pre-trained language model based on the enhanced annotated text data to obtain the initial business entity recognition model. This step, through a differentiated learning rate strategy, preserves the general semantic understanding capability of the pre-trained language model while enabling the top-level parameters to quickly adapt to the business entity recognition task.

[0062] The beneficial effect of this step is that by performing hierarchical fine-tuning training on the pre-trained language model based on full-word masking according to the enhanced labeled text data, an initial business entity recognition model is obtained. This model can retain the underlying general semantic knowledge while enabling the top-level task parameters to quickly adapt to the business entity recognition requirements, effectively improving the model's generalization ability in data-sparse scenarios.

[0063] In one feasible implementation, step S30 may include steps A31 to A36: Step A31: Obtain a pre-trained language model based on full-word masking; Understandably, step A31 refers to obtaining a pre-trained language model based on full-word masking, thereby providing a pre-trained foundation model with full-word masking capabilities for subsequent hierarchical fine-tuning training. In this embodiment, the pre-trained language model is specifically the BERT-wwm-ext model.

[0064] The beneficial effect of this step is that by obtaining a pre-trained language model based on whole-word masking, the whole-word masking technology can be used to mask complete words, improve the model's ability to understand word-level semantics, and provide a high-quality semantic representation foundation for business entity recognition.

[0065] Step A32: Determine the total number of hidden layers, the initial learning rate of the top layer, and the preset learning rate decay coefficient of the pre-trained language model; It should be noted that the total number of hidden layers refers to the total number of hidden layers in the pre-trained language model, that is, the total number of neural network layers used for feature extraction and semantic transformation in the model. The initial learning rate of the top layer refers to the initial learning rate of the top-level network of the pre-trained language model at the start of fine-tuning training, that is, the hyperparameter controlling the magnitude of the top-level parameter updates. The preset learning rate decay coefficient refers to a pre-set coefficient parameter used to decay the learning rate layer by layer, that is, a value controlling the degree to which the learning rate of each layer decays relative to the learning rate of the top layer.

[0066] Understandably, step A32 refers to determining the total number of hidden layers, the initial learning rate of the top layer, and the preset learning rate decay coefficient of the pre-trained language model, thereby providing basic parameters for subsequent calculation of the target learning rate of each layer.

[0067] The beneficial effect of this step is that by determining the total number of hidden layers, the initial learning rate of the top layer, and the preset learning rate decay coefficient of the pre-trained language model, it can provide accurate parameter basis for the layer-by-layer calculation of the learning rate, ensuring that the differentiated settings of the learning rate of each layer conform to the structural characteristics of the model.

[0068] Step A33: Based on the initial learning rate of the top layer and the preset learning rate decay coefficient, calculate the learning rate of each hidden layer layer by layer downwards to obtain the target learning rate of each layer; It should be noted that the target learning rate for each layer refers to the actual learning rate value used by each hidden layer, calculated layer by layer downwards based on the initial learning rate of the top layer and the preset learning rate decay coefficient. That is, the differentiated learning rate value with a smaller learning rate at the bottom layer and a larger learning rate at the top layer.

[0069] Understandably, step A33 refers to calculating the learning rate of each hidden layer layer by layer downwards based on the initial learning rate of the top layer and the preset learning rate decay coefficient, thereby obtaining the target learning rate of each layer.

[0070] The beneficial effect of this step is that, based on the initial learning rate of the top layer and the preset learning rate decay coefficient, the learning rate of each hidden layer is calculated layer by layer downwards to obtain the target learning rate of each layer. This enables a smooth decrease in the learning rate from the top layer to the bottom layer, keeping the parameters of the bottom layer network stable during fine-tuning.

[0071] Step A34: Calculate the learning rate of the word embedding layer based on the total number of hidden layers and the preset learning rate decay coefficient, and obtain the target learning rate of the word embedding layer; It should be noted that the word embedding layer refers to the input layer in a pre-trained language model that maps text words into low-dimensional dense vector representations; that is, the network layer responsible for converting discrete word symbols into continuous vector representations. The target learning rate of the word embedding layer refers to the actual learning rate used by the word embedding layer, calculated based on the total number of hidden layers and the preset learning rate decay coefficient; that is, the parameter update rate of the word embedding layer during fine-tuning training.

[0072] Understandably, step A34 refers to calculating the learning rate of the word embedding layer based on the total number of hidden layers and the preset learning rate decay coefficient, thereby obtaining the target learning rate of the word embedding layer.

[0073] The beneficial effect of this step is that by calculating the learning rate of the word embedding layer based on the total number of hidden layers and the preset learning rate decay coefficient, the target learning rate of the word embedding layer can be obtained. This allows the word embedding layer to update its parameters with a very small amplitude during the fine-tuning process, effectively preserving the general lexical semantic knowledge learned in the pre-training stage.

[0074] In the specific implementation, the learning rate of the hidden layers of the BERT model is set to decrease layer by layer, and the mathematical formula is as follows: The learning rate of the i-th hidden layer:

[0075] Word embedding layer learning rate:

[0076] Where L is the total number of hidden layers in the BERT model, η is the initial learning rate of the top layer, and γ is the learning rate decay coefficient.

[0077] The lower layers use a smaller learning rate to preserve general knowledge, while the upper layers use a larger learning rate to quickly adapt to the NER task. The learning rate decreases layer by layer from the top to the bottom hidden layers. This ensures that the lower-level encoder outputs high-quality contextual representations, with general knowledge adjusted slowly and task-specific knowledge adapted quickly in the upper layers.

[0078] Step A35: Based on the target learning rate of each layer and the target learning rate of the word embedding layer, perform differential updates on the parameters of each layer of the pre-trained language model to obtain the pre-trained language model with updated parameters; It should be noted that differentiated update refers to a method of adjusting and updating the parameters of different network layers in a pre-trained language model differently based on the target learning rate of each layer. In other words, different network layers use different learning rates for parameter optimization. The pre-trained language model after parameter updates refers to the base model whose parameters have been adjusted after the differentiated update process.

[0079] Understandably, step A35 refers to differentially updating the parameters of each layer of the pre-trained language model based on the target learning rate of each layer and the target learning rate of the word embedding layer, thereby obtaining the pre-trained language model with updated parameters.

[0080] The beneficial effect of this step is that by differentially updating the parameters of each layer of the pre-trained language model according to the target learning rate of each layer and the target learning rate of the word embedding layer, the pre-trained language model with updated parameters can be obtained. This can balance the preservation of general knowledge and the learning of task knowledge during the fine-tuning process, and prevent the underlying general semantic knowledge from being over-covered.

[0081] Step A36: Train the pre-trained language model with the updated parameters of the enhanced labeled text data for business entity recognition to obtain the initial business entity recognition model.

[0082] Understandably, step A36 refers to inputting the enhanced labeled text data into the pre-trained language model with updated parameters for business entity recognition training, thereby obtaining the initial business entity recognition model. This step utilizes business text data with special labels and boundary annotations to perform end-to-end entity recognition training on the pre-trained language model with updated parameters.

[0083] The beneficial effect of this step is that the pre-trained language model with the enhanced labeled text data input parameters is used to train business entity recognition, resulting in an initial business entity recognition model. This model can fully learn the entity boundary features and easily confused entity differentiation features in the business scenario, and has basic business entity recognition capabilities.

[0084] Step S40: The initial business entity recognition model is adversarially trained using an adversarial perturbation generation strategy based on model gradients to obtain the target business entity recognition model. It should be noted that the adversarial perturbation generation strategy refers to a training strategy that utilizes the model's own gradient information to generate small perturbation samples. That is, it superimposes small perturbations calculated based on the gradient direction onto the original input samples to construct difficult samples. Adversarial training refers to a training method that inputs adversarial perturbation samples into the model to enhance its robustness; that is, it actively introduces difficult samples to improve the model's resistance to noise and interference. The target business entity recognition model refers to the final business entity recognition model with enhanced robustness obtained after adversarial training.

[0085] Understandably, step S40 refers to adversarially training the initial business entity recognition model using an adversarial perturbation generation strategy based on model gradients, thereby obtaining the target business entity recognition model. This step utilizes the model's own gradient as a guide, actively introducing difficult samples during the training process to enhance the model's adaptability to noise, typos, and blurred boundary interference in the text after optical character recognition.

[0086] The beneficial effect of this step is that by using an adversarial perturbation generation strategy based on model gradient to adversarially train the initial business entity recognition model, the target business entity recognition model can be obtained. This enables the model to actively learn the noise distribution features and difficult sample patterns in the text after optical character recognition, significantly improving its robustness to interference factors such as noise, misspellings, and blurred boundaries.

[0087] In the specific implementation, the FGM adversarial training method is introduced, which uses the model's own gradient as a "guide" to add small perturbations during the training process, actively introduce difficult samples, and improve the robustness of the model.

[0088] Step S50: Input the target business text to be identified into the target business entity recognition model for entity recognition processing to obtain the business entity recognition result.

[0089] It should be noted that the target business text refers to the business text data to be processed for entity recognition, i.e., the original business text content input into the target business entity recognition model for recognition. Entity recognition processing refers to the process of automatically identifying and classifying business entities in the text using a trained model, i.e., determining the entity category and boundary position of each word in the text. The business entity recognition result refers to the final recognition output containing entity category and boundary position information obtained after entity recognition processing.

[0090] Understandably, step S50 refers to inputting the target business text to be identified into the target business entity recognition model for entity recognition processing, thereby obtaining the business entity recognition result. This step uses the trained target business entity recognition model to perform end-to-end entity category prediction and boundary localization on the business text to be identified.

[0091] The beneficial effect of this step is that by inputting the target business text to be identified into the target business entity recognition model for entity recognition processing, the business entity recognition result can be obtained. This enables high-precision automatic identification and boundary positioning of various business entities in the business text after optical character recognition, meeting the high compliance recognition requirements in financial and government scenarios.

[0092] In one feasible implementation, step S50 may include steps A41 to A44: Step A41: Obtain the target business text to be identified; Understandably, step A41 refers to acquiring the target business text to be identified, thereby providing input data for subsequent entity recognition processing. In this embodiment, the target business text is the business watermark text to be identified after optical character recognition.

[0093] The benefit of this step is that by acquiring the target business text to be identified, an accurate input data source can be provided for the entity recognition model, ensuring that the recognition process is carried out on the correct business text content.

[0094] Step A42: Input the target business text into the target business entity recognition model to obtain the entity label prediction probability of each word; It should be noted that the entity label prediction probability refers to the probability output value of the target business entity recognition model for each word in the text belonging to each entity label, that is, the numerical representation of the model's classification confidence for each word.

[0095] Understandably, step A42 refers to inputting the target business text into the target business entity recognition model to obtain the entity label prediction probability of each word.

[0096] The beneficial effect of this step is that by inputting the target business text into the target business entity recognition model to obtain the entity label prediction probability of each word, the semantic features and boundary features learned by the model can be used to perform a probabilistic evaluation of the entity affiliation of each word, providing a data foundation for subsequent category determination.

[0097] Step A43: Determine the entity category and boundary position of each word based on the entity label prediction probability and boundary constraints of each word. It should be noted that boundary constraints refer to the rules used to constrain and correct the results of entity boundary determination during entity recognition, i.e., the judgment criteria that ensure the start and end positions of entities conform to the annotation specifications. Entity category refers to the business type to which the entity identified in the business text belongs, i.e., business categories such as customer name, account manager name, and product name. Boundary position refers to the start and end positions of the entity in the text, i.e., the specific index range of the entity's span in the text sequence.

[0098] Understandably, step A43 refers to determining the entity category and boundary position of each word based on the entity label prediction probability and boundary constraints of each word.

[0099] The beneficial effect of this step is that it determines the entity category and boundary position of each word based on the entity label prediction probability and boundary constraints of each word. It can effectively improve the accuracy of entity boundary localization and the reliability of entity category determination by combining the dual constraints of model prediction probability and boundary rules.

[0100] Step A44: Generate business entity recognition results based on the entity category and boundary position of each word.

[0101] Understandably, step A44 refers to generating a business entity recognition result based on the entity category of each word and the boundary position, thereby outputting a final recognition result containing entity category and boundary position information.

[0102] The beneficial effect of this step is that it generates business entity recognition results based on the entity category and boundary position of each word, and can integrate the discrete word information identified by the model into a complete entity recognition output, forming a structured recognition result that can be directly used for business applications.

[0103] This embodiment provides a business entity recognition method. By combining data augmentation and special labeling, the training sample size is expanded and the boundary differentiation information of easily confused entities is strengthened. A layer-wise decreasing learning rate strategy is used to fine-tune the pre-trained language model based on full-word masking. This preserves the general semantic knowledge at the lower level while enabling the top-level task parameters to quickly adapt to the business entity recognition task, effectively improving the model's generalization ability in data-sparse scenarios. Adversarial training is performed using an adversarial perturbation generation strategy based on model gradients. This allows the model to actively learn the noise features and distribution of difficult samples in the text after optical character recognition, significantly enhancing its robustness to noise, typos, and blurred boundary interference. In summary, these methods achieve high-precision boundary recognition and accurate category differentiation of business entities in watermarked text after optical character recognition.

[0104] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 Step S40 includes steps S401 to S405: Step S401: Input the current training sample into the initial business entity recognition model for forward calculation to obtain the predicted label; It should be noted that the current training sample refers to the sample data input to the model for learning in the current batch during model training, that is, the business text sample extracted from the augmented labeled text data for this parameter update. The predicted label refers to the entity category prediction result output by the initial business entity recognition model after performing forward computation on each word in the current training sample; that is, the distribution information of the entity labels to which each word belongs, inferred by the model. Forward computation refers to the computational process of passing input data layer by layer from the input layer to the output layer and generating prediction results; that is, the forward propagation operation of data from input to output in the neural network model.

[0105] Understandably, step S401 refers to inputting the current training sample into the initial business entity recognition model for forward computation to obtain the predicted label. This step, through forward propagation, enables the initial business entity recognition model to predict the entity category of each word in the current training sample.

[0106] The beneficial effect of this step is that by inputting the current training samples into the initial business entity recognition model for forward computation to obtain the predicted labels, it can provide the model prediction output for subsequent loss function calculation, thereby quantifying the model error.

[0107] Step S402: Calculate the loss function value using the joint loss function based on the predicted label and the true label of the current training sample; It should be noted that the true labels refer to the pre-labeled correct entity category information for each word in the current training samples, i.e., the standard answer annotation data used to supervise model training. The joint loss function is a comprehensive loss metric function composed of multiple loss calculation methods; it is a composite loss calculation method that simultaneously considers entity boundary accuracy and prediction confidence. The loss function value is the numerical value obtained by calculating the difference between the model's prediction and the true labels through the joint loss function; it is a quantified error metric that guides the direction and magnitude of model parameter updates.

[0108] Understandably, step S402 refers to calculating the loss function value using a joint loss function based on the predicted labels and the true labels of the current training samples. This step comprehensively compares the model's prediction results with the true labels using the joint loss function to obtain an error metric used to guide parameter optimization.

[0109] The beneficial effect of this step is that by calculating the loss function value through a joint loss function based on the predicted label and the true label of the current training sample, the entity boundary accuracy and prediction confidence can be considered simultaneously, providing comprehensive error feedback for model parameter optimization.

[0110] In one feasible implementation, step S402 may include steps A51 to A54: Step A51: Calculate the boundary constraint loss value using the boundary constraint loss function based on the entity boundary information of each word in the predicted and true labels; It should be noted that the boundary constraint loss function refers to the loss calculation function designed for the accuracy of entity boundary recognition, that is, the boundary optimization loss that enhances the model's ability to distinguish the starting and ending positions of entities. The boundary constraint loss value is the loss value calculated by the boundary constraint loss function that reflects the accuracy of entity boundary prediction, that is, a quantitative indicator for measuring the model's boundary localization error.

[0111] Understandably, step A51 refers to calculating the boundary constraint loss value using the boundary constraint loss function based on the entity boundary information of each word in the predicted and true labels. This step takes entity boundary recognition as the optimization objective and calculates the deviation loss between the predicted boundary and the true boundary.

[0112] The beneficial effect of this step is that by calculating the boundary constraint loss value through the boundary constraint loss function based on the entity boundary information of each word in the predicted label and the real label, the model's ability to perceive the starting and ending positions of entities can be strengthened, and the localization accuracy of entity boundaries can be significantly improved.

[0113] Step A52: Based on the predicted labels and the preset smoothing coefficient, smooth the predicted labels at the word level to obtain the smoothed predicted labels; It's important to note that the preset smoothing coefficient refers to a pre-defined parameter used to smooth the predicted labels, essentially a softening factor to prevent the model from overconfident in its predictions. The lexical level refers to the level where the processing granularity is based on the smallest semantic unit in the text, i.e., the word, meaning each individual word is processed individually. Smoothing refers to the process of softening and adjusting the probability distribution of the predicted labels using the preset smoothing coefficient; this is a regularization operation that reduces extreme probability values ​​and enhances the uniformity of the probability distribution. The smoothed predicted labels refer to the prediction results with a more uniform probability distribution obtained after smoothing, thus avoiding overconfident predictions and a softened label distribution.

[0114] Understandably, step A52 refers to smoothing the predicted labels at the word level based on the predicted labels and a preset smoothing coefficient, resulting in smoothed predicted labels. This step softens and adjusts the predicted probability of each word using a preset smoothing coefficient, making the prediction distribution more uniform.

[0115] The beneficial effect of this step is that by smoothing the predicted labels at the word level based on the predicted labels and the preset smoothing coefficient, the smoothed predicted labels can be obtained. This can effectively prevent the model from being overconfident in its predictions, play a regularization role, and improve the model's generalization ability.

[0116] Step A53: Calculate the smoothing loss value based on the smoothed predicted labels and the true labels; It should be noted that the smoothing loss value refers to the loss value calculated based on the smoothed predicted label and the true label, that is, a metric that reflects the difference between the smoothed prediction and the true label.

[0117] As is understandable, step A53 refers to calculating the smoothed loss value based on the smoothed predicted labels and the true labels. This step compares the smoothed predicted labels with the true labels to obtain the smoothed loss metric.

[0118] The benefit of this step is that by calculating the smoothing loss value based on the smoothed predicted labels and the true labels, the loss can be calculated on the smoothed predicted distribution, thus avoiding overfitting caused by the model's overconfidence in its predictions.

[0119] Step A54: Perform a weighted summation of the boundary constraint loss value and the smoothing loss value to obtain the loss function value.

[0120] Understandably, step A54 refers to the weighted summation of the boundary constraint loss value and the smoothing loss value to obtain the loss function value. This step fuses the boundary constraint loss value and the smoothing loss value according to preset weights to form the final joint loss metric.

[0121] The beneficial effect of this step is that by weighting and summing the boundary constraint loss value and the smoothing loss value to obtain the loss function value, the model can take into account both boundary accuracy and prediction stability during the optimization process, resulting in a more robust loss metric.

[0122] Step S403: Calculate the gradient values ​​of each parameter of the initial business entity recognition model based on the loss function value; It should be noted that the gradient value refers to the partial derivative obtained by taking the loss function value with respect to each parameter of the initial business entity identification model, which is a sensitivity index that indicates the direction and magnitude of parameter updates.

[0123] Understandably, step S403 refers to calculating the gradient values ​​of each parameter of the initial business entity recognition model based on the loss function value. This step uses the backpropagation algorithm to propagate the loss function value back layer by layer and calculate the partial derivatives of each parameter.

[0124] The beneficial effect of this step is that by calculating the gradient values ​​of each parameter of the initial business entity identification model based on the loss function value, it is possible to clarify the contribution of each parameter to the current prediction error, and provide directional guidance for subsequent adversarial perturbation generation and parameter updates.

[0125] Step S404: Generate adversarial perturbation samples based on gradient values ​​and preset perturbation coefficients; It should be noted that the preset perturbation coefficient refers to a pre-set proportional coefficient used to control the magnitude of adversarial perturbations, that is, an intensity parameter that adjusts the degree of deviation between the adversarial samples and the original samples. Adversarial perturbation samples refer to training samples with slight perturbations generated based on gradient values ​​and preset perturbation coefficients, that is, adversarial input data that makes prediction difficult for the model.

[0126] Understandably, step S404 refers to generating adversarial perturbation samples based on the gradient values ​​and preset perturbation coefficients. This step utilizes the model's own gradient information as a guide to add small perturbations along the gradient direction near the original samples.

[0127] The beneficial effect of this step is that it generates adversarial perturbation samples based on gradient values ​​and preset perturbation coefficients, which can proactively construct difficult samples in business scenarios, allowing the model to be exposed to and learn noise interference features in advance during the training process, thereby enhancing its adaptability to noise and typos in the text after optical character recognition.

[0128] Step S405: Input the anti-disturbance sample into the initial business entity recognition model for training to obtain the target business entity recognition model.

[0129] Understandably, step S405 refers to inputting adversarial perturbation samples into the initial business entity recognition model for training, thereby obtaining the target business entity recognition model. This step improves the robustness of the model through joint training with adversarial samples and original samples by using adversarial perturbation samples as additional training data to participate in model optimization.

[0130] The beneficial effect of this step is that by inputting adversarial perturbation samples into the initial business entity recognition model for training to obtain the target business entity recognition model, the model can learn to resist minor perturbations during adversarial training, significantly improving the stability and accuracy of recognizing noise, typos, and blurred boundary interference in the text after optical character recognition.

[0131] In the implementation, for scenarios with unclear entity boundaries, the identification of each entity category is treated as a binary classification segmentation problem. The Dice Loss function is used, and Label Smoothing regularization is introduced to prevent overfitting. Label Smoothing optimizes at the token level, ensuring that each individual token is correctly classified, while also acting as a regularization mechanism by penalizing overconfident predictions, thus improving the model's generalization ability. The complete loss function is as follows:

[0132] This embodiment provides a business entity recognition method. Through data augmentation and special labeling, it expands the training samples and enhances the ability to distinguish the boundaries of easily confused entities. A hierarchical learning rate strategy is used to fine-tune the pre-trained language model, allowing the top-level parameters to quickly adapt to the business entity recognition task while preserving general semantic knowledge. Adversarial training is performed using an adversarial perturbation generation strategy based on model gradients, supplemented by joint loss function optimization. This significantly enhances the model's robustness to noise, typos, and blurred boundary interference in the text after optical character recognition, achieving high-precision boundary recognition and accurate category distinction of business entities.

[0133] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the business entity identification method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0134] This application also provides a business entity identification device, please refer to... Figure 3 The business entity identification device includes: The data augmentation annotation module 10 is used to acquire the business watermark text after optical character recognition, perform data augmentation processing on the business watermark text, and obtain the enhanced business text. The special labeling module 20 is used to perform special labeling processing on the enhanced business text to obtain enhanced labeled text data; The hierarchical fine-tuning module 30 is used to perform hierarchical fine-tuning training on the pre-trained language model based on the enhanced labeled text data to obtain the initial business entity recognition model. The adversarial training module 40 is used to perform adversarial training on the initial business entity recognition model through an adversarial perturbation generation strategy based on model gradient, so as to obtain the target business entity recognition model. The entity recognition module 50 is used to input the target business text to be recognized into the target business entity recognition model for entity recognition processing, and obtain the business entity recognition result.

[0135] The business entity recognition device provided in this application, employing the business entity recognition method described in the above embodiments, can solve the technical problem of how to perform business entity recognition on business watermark text after optical character recognition. Compared with the prior art, the beneficial effects of the business entity recognition device provided in this application are the same as those of the business entity recognition method provided in the above embodiments, and other technical features in the business entity recognition device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0136] In one embodiment, the data augmentation annotation module 10 is also used to acquire the business watermark text after optical character recognition; The business watermark text is divided into a training set and a verification set according to a preset ratio; The non-core entities in the training set are replaced with synonyms to obtain the first replacement text. The entities in the first replacement text are replaced to generate a virtual card number identifier and a virtual identity identifier, thus obtaining the second replacement text; The second replacement text is processed by initial and intermediate non-entity annotation to obtain the enhanced business text.

[0137] In one embodiment, the special labeling module 20 is also used to identify easily confused entity pairs in the enhanced business text, the easily confused entity pairs including a first easily confused entity and a second easily confused entity; Add a first special tag to the first easily confused entity; Add a second special tag to the second easily confused entity; Based on the first special label and the second special label, boundary distinction annotations are performed on the first and second easily confused entities to obtain enhanced labeled text data.

[0138] In one embodiment, the hierarchical fine-tuning module 30 is also used to obtain a pre-trained language model based on a full-word mask; Determine the total number of hidden layers, the initial learning rate of the top layer, and the preset learning rate decay coefficient of the pre-trained language model; Based on the initial learning rate of the top layer and the preset learning rate decay coefficient, the learning rate of each hidden layer is calculated layer by layer downwards to obtain the target learning rate of each layer; The learning rate of the word embedding layer is calculated based on the total number of hidden layers and the preset learning rate decay coefficient, thus obtaining the target learning rate of the word embedding layer. Based on the target learning rate of each layer and the target learning rate of the word embedding layer, the parameters of each layer of the pre-trained language model are updated differentially to obtain the pre-trained language model with updated parameters. The pre-trained language model with updated input parameters of the enhanced labeled text data is used to train business entity recognition, resulting in an initial business entity recognition model.

[0139] In one embodiment, the adversarial training module 40 is further configured to input the current training sample into the initial business entity recognition model for forward calculation to obtain the predicted label; The loss function value is calculated using a joint loss function based on the predicted label and the true label of the current training sample. Based on the loss function value, calculate the gradient values ​​of each parameter of the initial business entity recognition model; Generate adversarial samples based on gradient values ​​and preset perturbation coefficients; The initial business entity recognition model is trained by inputting the adversarial perturbation sample, and the target business entity recognition model is obtained.

[0140] In one embodiment, the adversarial training module 40 is further configured to calculate the boundary constraint loss value using a boundary constraint loss function based on the entity boundary information of each word in the predicted label and the real label. Based on the predicted labels and the preset smoothing coefficient, the predicted labels are smoothed at the word level to obtain the smoothed predicted labels. Calculate the smoothing loss value based on the smoothed predicted labels and the true labels; The loss function value is obtained by weighted summation of the boundary constraint loss value and the smoothing loss value.

[0141] In one embodiment, the entity recognition module 50 is further configured to acquire the target business text to be recognized; Input the target business text into the target business entity recognition model to obtain the entity label prediction probability of each word; Based on the entity label prediction probability and boundary constraints of each word, the entity category and boundary position of each word are determined. Based on the entity category and boundary position of each word, business entity recognition results are generated.

[0142] This application provides a business entity identification device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the business entity identification method in Embodiment 1 above.

[0143] The following is for reference. Figure 4 This document illustrates a structural diagram of a business entity identification device suitable for implementing embodiments of this application. The business entity identification device in these embodiments may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 4 The business entity identification device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0144] like Figure 4As shown, the business entity identification device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the business entity identification device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the business entity identification device to communicate wirelessly or wiredly with other devices to exchange data. While the figure shows business entity identification devices with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0145] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0146] The business entity recognition device provided in this application, employing the business entity recognition method described in the above embodiments, can solve the technical problem of how to perform business entity recognition on business watermark text after optical character recognition. Compared with the prior art, the beneficial effects of the business entity recognition device provided in this application are the same as those of the business entity recognition device method provided in the above embodiments, and other technical features in this business entity recognition device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0147] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0148] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0149] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the business entity identification method in the above embodiments.

[0150] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0151] The aforementioned computer-readable storage medium may be included in the business entity identification device; or it may exist independently and not be assembled into the business entity identification device.

[0152] The aforementioned computer-readable storage medium carries one or more programs that, when executed by the business entity recognition device, cause the following to occur: acquire the business watermark text after optical character recognition; perform data augmentation processing on the business watermark text to obtain enhanced business text; perform special labeling processing on the enhanced business text to obtain enhanced labeled text data; perform hierarchical fine-tuning training on a pre-trained language model based on full-word masking based on the enhanced labeled text data to obtain an initial business entity recognition model; perform adversarial training on the initial business entity recognition model through an adversarial perturbation generation strategy based on model gradients to obtain a target business entity recognition model; and input the target business text to be recognized into the target business entity recognition model for entity recognition processing to obtain the business entity recognition result.

[0153] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0154] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0155] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0156] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described business entity recognition method, and can solve the technical problem of how to perform business entity recognition on business watermark text after optical character recognition. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the business entity recognition method provided in the above embodiments, and will not be repeated here.

[0157] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the business entity identification method described above.

[0158] The computer program product provided in this application can solve the technical problem of how to perform business entity recognition on business watermark text after optical character recognition. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the business entity recognition method provided in the above embodiments, and will not be repeated here.

[0159] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A business entity identification method, characterized by, The business entity identification method includes: Obtain the business watermark text after optical character recognition, and perform data enhancement processing on the business watermark text to obtain enhanced business text; The enhanced business text is labeled with special tags to obtain enhanced labeled text data; Based on the enhanced annotated text data, the pre-trained language model based on full-word masking is subjected to hierarchical fine-tuning training to obtain the initial business entity recognition model; The initial business entity identification model is adversarially trained using an adversarial perturbation generation strategy based on model gradients to obtain the target business entity identification model. The target business text to be identified is input into the target business entity recognition model for entity recognition processing to obtain the business entity recognition result.

2. The method as described in claim 1, characterized in that, The steps of obtaining the business watermark text after optical character recognition and performing data enhancement processing on the business watermark text to obtain enhanced business text include: Obtain the business watermark text after optical character recognition; The watermarked text is divided into a training set and a verification set according to a preset ratio; The non-core entities in the training set are subjected to synonym replacement processing to obtain the first replacement text; The entities in the first replacement text are replaced to generate a virtual card number identifier and a virtual identity identifier, thus obtaining the second replacement text; The second replacement text is processed by initial and intermediate non-entity annotation to obtain the enhanced business text.

3. The method as described in claim 1, characterized in that, The step of performing special tagging processing on the enhanced business text to obtain enhanced tagged text data includes: Identify easily confused entity pairs in the enhanced business text, wherein the easily confused entity pairs include a first easily confused entity and a second easily confused entity; Add a first special tag to the first easily confused entity; Add a second special tag to the second easily confused entity; Based on the first special label and the second special label, the first easily confused entity and the second easily confused entity are marked with boundary distinction to obtain enhanced labeled text data.

4. The method as described in claim 1, characterized in that, The step of performing hierarchical fine-tuning training on the pre-trained language model based on the enhanced annotated text data to obtain the initial business entity recognition model includes: Obtain a pre-trained language model based on full-word masking; Determine the total number of hidden layers, the initial learning rate of the top layer, and the preset learning rate decay coefficient of the pre-trained language model; Based on the initial learning rate of the top layer and the preset learning rate decay coefficient, the learning rate of each hidden layer is calculated layer by layer downwards to obtain the target learning rate of each layer; The learning rate of the word embedding layer is calculated based on the total number of hidden layers and the preset learning rate decay coefficient, and the target learning rate of the word embedding layer is obtained. Based on the target learning rate of each layer and the target learning rate of the word embedding layer, the parameters of each layer of the pre-trained language model are updated differentially to obtain the pre-trained language model with updated parameters. The enhanced labeled text data is input into the pre-trained language model with updated parameters to train business entity recognition, thereby obtaining the initial business entity recognition model.

5. The method as described in claim 1, characterized in that, The step of performing adversarial training on the initial business entity recognition model using an adversarial perturbation generation strategy based on model gradients to obtain the target business entity recognition model includes: The current training sample is input into the initial business entity recognition model for forward calculation to obtain the predicted label; Based on the predicted label and the true label of the current training sample, the loss function value is calculated using a joint loss function; Based on the loss function value, calculate the gradient values ​​of each parameter of the initial business entity identification model; Based on the gradient value and the preset perturbation coefficient, generate adversarial perturbation samples; The adversarial perturbation sample is input into the initial business entity recognition model for training to obtain the target business entity recognition model.

6. The method as described in claim 5, characterized in that, The step of calculating the loss function value based on the predicted label and the true label of the current training sample using a joint loss function includes: Based on the entity boundary information of each word in the predicted label and the real label, the boundary constraint loss value is calculated using the boundary constraint loss function; Based on the predicted label and the preset smoothing coefficient, the predicted label is smoothed at the word level to obtain the smoothed predicted label. Calculate the smoothing loss value based on the smoothed predicted label and the true label; The loss function value is obtained by weighted summation of the boundary constraint loss value and the smoothing loss value.

7. The method as described in claim 1, characterized in that, The step of inputting the target business text to be identified into the target business entity recognition model for entity recognition processing to obtain the business entity recognition result includes: Obtain the target business text to be identified; The target business text is input into the target business entity recognition model to obtain the entity label prediction probability of each word. Based on the entity label prediction probability and boundary constraints of each word, the entity category and boundary position of each word are determined. Based on the entity category of each word and the boundary position, a business entity identification result is generated.

8. A business entity identification device, characterized in that, The device includes: The data augmentation annotation module is used to acquire the business watermark text after optical character recognition, and to perform data augmentation processing on the business watermark text to obtain the enhanced business text. The special labeling module is used to perform special labeling processing on the enhanced business text to obtain enhanced labeled text data; The hierarchical fine-tuning module is used to perform hierarchical fine-tuning training on the pre-trained language model based on the enhanced labeled text data to obtain the initial business entity recognition model. The adversarial training module is used to perform adversarial training on the initial business entity recognition model through an adversarial perturbation generation strategy based on model gradients, so as to obtain the target business entity recognition model. The entity recognition module is used to input the target business text to be recognized into the target business entity recognition model for entity recognition processing, and obtain the business entity recognition result.

9. A business entity identification device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the business entity identification method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the business entity identification method as described in any one of claims 1 to 7.

Images