Business named entity extraction method, device, equipment and readable storage medium
By obtaining image collections from unstructured data, deduplication and denoising of named entities are performed, and the problems of noise and redundancy in business named entities are solved, improving data quality and accuracy.
Patent Information
- Application Number
- CN202210212270.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-04
- Publication Date
- 2025-02-07
- Estimated Expiration
- 2042-03-04
AI Technical Summary
Business named entities extracted from unstructured data are noise and redundant, and the data quality is low and difficult to effectively utilize.
By obtaining the image set, naming entity extraction is performed based on each image, the clustering algorithm processes the named entity set, the voting method deduplication, and finally performs denoising to obtain the standard named entity set.
Improve the accuracy and quality of the results of business named entity extraction, reduce noise interference through verification and denoising processing, and enhance the reliability of named entities.
Smart Images
Figure CN114564961B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a method, device, equipment and readable storage medium for extracting business named entities. Background Art
[0002] With the development of AI technology, information that was previously not effectively used has gradually been valued, such as unstructured data such as speech and images. One of the main ways to obtain effective structured information from unstructured data is named entity extraction. However, the entities extracted from texts from these special sources contain noise and redundancy, and the data quality is low. Summary of the invention
[0003] The purpose of the present invention is to provide a method, device, equipment and readable storage medium for extracting business named entities to improve the above-mentioned problems.
[0004] In order to achieve the above objectives, the present application provides the following technical solutions:
[0005] On the one hand, an embodiment of the present application provides a method for extracting business named entities, the method comprising:
[0006] Acquire an image set, where the image set is a set of images containing business information generated when processing the business;
[0007] Based on each image in the image set, obtaining a named entity corresponding to each image, and grouping the named entities to obtain a named entity set;
[0008] Processing the named entity set based on a clustering algorithm to obtain a named entity clustering cluster division result;
[0009] Processing the named entity clustering results based on a voting method to obtain a standard named entity set;
[0010] The standard named entity set is subjected to denoising processing to obtain a named entity extraction result of the business.
[0011] Optionally, obtaining a named entity corresponding to each image based on each image in the image set, and grouping the named entities to obtain a named entity set, includes:
[0012] Obtaining OCR text of each image based on each of the images;
[0013] Named entity extraction is performed on the OCR text of each image, and the extracted named entities are grouped to obtain the named entity set.
[0014] Optionally, the voting-based method is used to process the named entity clustering result to obtain a standard named entity set, including:
[0015] Based on the named entity clustering result, each cluster is deduplicated using a voting method to obtain a standard named entity corresponding to each cluster;
[0016] The standard named entities corresponding to each cluster are collected to obtain a first standard named entity set;
[0017] The first standard named entity set is analyzed, wherein, if one of the standard named entities in the first standard named entity set is a substring of another standard named entity, the one standard named entity is removed from the first standard named entity set to obtain the standard named entity set.
[0018] Optionally, based on the named entity clustering cluster division result, each cluster is deduplicated by using a voting method to obtain a standard named entity corresponding to each cluster, including:
[0019] Based on the named entity clustering result, traverse and calculate the sum of similarities between each named entity and other named entities in the cluster to obtain the sum of similarities of each named entity in each cluster;
[0020] The sum of similarities of each of the named entities in each cluster is arranged in descending order, and the named entity with the highest sum of similarities is taken as the standard named entity of each cluster.
[0021] Optionally, based on the named entity clustering result, traversing and calculating the sum of similarities between each named entity and other named entities in the cluster to obtain the sum of similarities of each named entity in each cluster includes:
[0022] Calculating the edit distance between one of the named entities and another of the named entities in each cluster, and calculating the sum of the string lengths of one of the named entities and another of the named entities in each cluster;
[0023] The similarity between one of the named entities and another of the named entities is obtained based on the sum of the edit distance and the character string length.
[0024] Optionally, the performing denoising processing on the standard named entity set to obtain a named entity extraction result of the business includes:
[0025] Obtain denoising rules input by staff;
[0026] De-noising is performed on each of the standard named entities in the standard named entity set according to the de-noising rule to obtain a named entity extraction result of the business.
[0027] In a second aspect, an embodiment of the present application provides a device for extracting business named entities, the device comprising an acquisition module, a first calculation module, a second calculation module, a third calculation module and a denoising module.
[0028] An acquisition module, used to acquire an image set, wherein the image set is a set of images containing business information generated when processing the business;
[0029] A first calculation module, configured to obtain a named entity corresponding to each image based on each image in the image set, and to group the named entities to obtain a named entity set;
[0030] A second computing module is used to process the named entity set based on a clustering algorithm to obtain a named entity clustering cluster division result;
[0031] A third calculation module is used to process the named entity clustering result based on a voting method to obtain a standard named entity set;
[0032] The denoising module is used to perform denoising processing on the standard named entity set to obtain the named entity extraction result of the business.
[0033] Optionally, the first computing module includes:
[0034] A first computing unit, configured to obtain an OCR text of each image based on each of the images;
[0035] The extraction unit is used to extract named entities from the OCR text of each image, and to group the extracted named entities to obtain the named entity group.
[0036] Optionally, the third computing module includes:
[0037] A deduplication unit, configured to perform deduplication processing on each cluster by using a voting method based on the named entity clustering cluster division result, so as to obtain a standard named entity corresponding to each cluster;
[0038] A collection unit, used for collecting the standard named entities corresponding to each cluster to obtain a first standard named entity collection;
[0039] An analysis unit is used to analyze the first standard named entity set, wherein if one of the standard named entities in the first standard named entity set is a substring of another standard named entity, then the one standard named entity is removed from the first standard named entity set to obtain the standard named entity set.
[0040] Optionally, the deduplication unit comprises:
[0041] A second calculation unit is used to traverse and calculate the sum of similarities between each of the named entities and other named entities in the cluster based on the named entity clustering result, so as to obtain the sum of similarities of each of the named entities in each cluster;
[0042] The sorting unit is used to arrange the sum of similarities of each of the named entities in each cluster in descending order, and take the named entity with the highest sum of similarities as the standard named entity of each cluster.
[0043] Optionally, the second computing unit includes:
[0044] A third calculation unit is used to calculate the edit distance between one of the named entities and another of the named entities in each cluster, and calculate the sum of the string lengths of one of the named entities and another of the named entities in each cluster;
[0045] The fourth calculation unit is used to obtain the similarity between one of the named entities and another of the named entities based on the sum of the edit distance and the character string length.
[0046] Optionally, the denoising module includes:
[0047] An acquisition unit, used for acquiring denoising rules input by a staff member;
[0048] A denoising unit is used to perform denoising processing on each of the standard named entities in the standard named entity set according to the denoising rule to obtain a named entity extraction result of the business.
[0049] In a third aspect, an embodiment of the present application provides a device for extracting business named entities, the device comprising a memory and a processor. The memory is used to store a computer program; the processor is used to implement the steps of the above-mentioned method for extracting business named entities when executing the computer program.
[0050] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned method for extracting business named entities are implemented.
[0051] The beneficial effects of the present invention are:
[0052] 1. The present invention can take multiple images in a business as input. The image information in the same business can verify the reliability of the extracted named entity information with each other, while the content of a single image is easily affected by noise. Therefore, the method of the present invention can improve the accuracy of the named entity extraction result of the final business.
[0053] 2. The present invention improves an algorithm suitable for English character clustering based on DBSCAN. The algorithm does not need to set the number of categories and random initial values in advance, the clustering results are unbiased, and the ability to resist noise interference is strong.
[0054] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or be understood by implementing the embodiments of the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.
[0056] Figure 1 It is a flowchart of a method for extracting business named entities described in an embodiment of the present invention;
[0057] Figure 2 It is a schematic diagram of the structure of a device for extracting business named entities described in an embodiment of the present invention;
[0058] Figure 3 It is a schematic diagram of the structure of a device for extracting business named entities described in an embodiment of the present invention. DETAILED DESCRIPTION
[0059] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0060] It should be noted that similar reference numerals or letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. At the same time, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.
[0061] Example 1
[0062] like Figure 1 As shown, this embodiment provides a method for extracting business named entities, which includes step S1, step S2, step S3, step S4 and step S5.
[0063] Step S1, obtaining an image set, wherein the image set is a set of images containing business information generated when processing the business;
[0064] Step S2, obtaining a named entity corresponding to each image based on each image in the image set, and grouping the named entities to obtain a named entity set;
[0065] Step S3, processing the named entity set based on a clustering algorithm to obtain a named entity clustering cluster division result;
[0066] Step S4, processing the named entity clustering result based on a voting method to obtain a standard named entity set;
[0067] Step S5: performing denoising processing on the standard named entity set to obtain a named entity extraction result of the business.
[0068] In this embodiment, multiple images in a business transaction can be used as input. The image information in the same business transaction can verify the reliability of the extracted named entity information. The content of a single image is easily affected by noise. The method in this embodiment can improve the accuracy of the named entity extraction result of the final business.
[0069] Merge the named entities extracted from all images. There is correlation between named entities in different images in the same business, so merging them when performing named entity denoising and deduplication is beneficial to improve accuracy. For example, multiple images in a business include business vouchers, invoices, contracts, etc. The contract number recorded in the business voucher and the contract number in the contract can be verified with each other; the two parties in the contract can correspond one by one with the payer and payee of the invoice and the buyer and seller in the business voucher. The more valid information of the same named entity, the better the deduplication and denoising effect.
[0070] This embodiment improves an algorithm suitable for English character clustering based on DBSCAN. The algorithm does not need to set the number of categories and random initial values in advance, and the clustering result is unbiased and has strong anti-noise interference ability. The named entity set is processed based on the clustering algorithm to obtain the named entity clustering cluster division result, which is specifically:
[0071] Step (1), input the named entity set S = {e1, e2, ..., e M}, similarity threshold epsilon, minimum number of neighbors of the central named entity MinNeighbors;
[0072] Step (2): Initialize the central named entity set Unvisited named entity set Γ = S, named entity clustering cluster The number of clusters k = 0;
[0073] Step (3): Find all the central named entities from the named entity set S. For any named entity name e i :
[0074] a) Use the similarity calculation formula to calculate each entity e i The set of similar named entities N i (For any other named entity e j ∈(Se i ), which is related to the named entity e i Similarity sim ij >epsilon, add it to the set N i )
[0075] b) If the number of elements in the similar named entity set | N i |≥MinNeighbors, name the entity name e i Join the central named entity set Ω=Ω∪{e i}
[0076] Step (4), if the central named entity set The algorithm ends, otherwise go to step (5);
[0077] Step (5): Randomly select a central named entity e from the central named entity set Ω and initialize the central named entity queue Q of the current cluster. cur ={e}, initialize category number k=k+1, initialize the current named entity cluster set C k ={e}, update the unvisited named entity set Γ=Γ-{e};
[0078] Step (6), update the central named entity set Ω = Ω - C k , if the current cluster center named entity queue Current cluster C k Generation is complete, update the named entity clustering cluster C = C ∪ {C k}, go to step (4);
[0079] Step (7), name the entity queue Q at the center of the current cluster cur Take out an element e′, Q cur =Q cur -e′, find all adjacent named entity sets N whose similarity to e′ is greater than the threshold epsilon e′ , remove the entities that have been clustered in the adjacent named entity set Δ=Γ∩N e′ , update the current named entity cluster set C k =C k ∩Δ, update the unvisited named entity set Γ = Γ-Δ, add the visited adjacent named entities to the current queue, and update Q cur =Q cur ∪(Δ∩Ω), go to step (6);
[0080] Step (8), output the named entity clustering result C = {C1, C2, ..., C K}.
[0081] In this embodiment, the minimum number of neighbors MinNeighbors is set to preliminarily filter out the noise that cannot be clustered.
[0082] In a specific implementation of the present disclosure, the step S2 may further include step S21 and step S22.
[0083] Step S21, obtaining the OCR text of each image based on each of the images;
[0084] Step S22: extract named entities from the OCR text of each image, and group the extracted named entities to obtain the named entity group.
[0085] Nowadays, the accuracy of OCR recognition has reached a very high level. For fixed-format images, the named entity can be obtained by directly locating the text slice where the named entity is located according to the OCR recognition coordinates. However, for a business with multiple types of images without fixed formats, it is difficult to locate the named entity slice. At this time, the slice texts after the image OCR are arranged from top to bottom and from left to right according to the slice coordinate position to obtain the OCR text corresponding to this image. Although the slice texts in the image are not continuous, the slice texts of the keywords and the corresponding values are still contextual to each other, and the splicing of slice texts can enhance their semantic relationship.
[0086] After obtaining the OCR text of each image, named entities are extracted from it in turn. The entity extraction model used in this embodiment is Bert+CRF. Bert is an open source English pre-trained language model from Google. The pre-trained language model is trained using massive text. The named entity extraction model is trained based on Bert. Since some parameters of feature extraction are fixed, the training can be accelerated and the model can converge faster. In addition, since the sample size of the pre-trained language model is very large, it can have a better reasoning effect. Finally, the pre-trained language model can also reduce the demand for training data.
[0087] When the OCR text of each image is input into the entity extraction model, the OCR text is input in the form of a sliding window to obtain the named entity corresponding to each image, as follows:
[0088] If the number of OCR texts preceding an OCR text is greater than or equal to 3, when calculating the named entity corresponding to the OCR text, the OCR text and the preceding 3 OCR texts and the following 3 OCR texts of the OCR text are simultaneously input into the entity extraction model;
[0089] If the number of OCR texts before an OCR text is greater than 0 and less than 3, when calculating the named entity corresponding to the OCR text, the OCR text, all the OCR texts before the OCR text, and the 3 OCR texts after the OCR text are simultaneously input into the entity extraction model;
[0090] If the number of OCR texts before an OCR text is 0, when calculating the named entity corresponding to this OCR text, this OCR text and the three OCR texts after this OCR text are simultaneously input into the entity extraction model;
[0091] Among them, the sorting method between OCR texts can be customized according to user needs.
[0092] In addition, in addition to the Bert+CRF model in this embodiment, an LSTM+CRF model can also be used for named entity extraction.
[0093] In a specific implementation of the present disclosure, the step S4 may further include step S41, step S42 and step S43.
[0094] Step S41: Based on the named entity clustering result, each cluster is deduplicated using a voting method to obtain a standard named entity corresponding to each cluster;
[0095] Step S42, grouping the standard named entities corresponding to each cluster to obtain a first standard named entity group;
[0096] Step S43, analyzing the first standard named entity set, wherein, if one of the standard named entities in the first standard named entity set is a substring of another standard named entity, then removing the one of the standard named entities from the first standard named entity set to obtain the standard named entity set.
[0097] In this embodiment, due to the completeness of the OCR slice, the named entity string will be divided into two or even more different slices. For example, "CHINA CITIC BANK BEIJING BRANCH" may be divided into two slices of "CHINA CITICBANK" and "BEIJING BRANCH". In order to eliminate such errors, substring judgment between standard named entities is performed after clustering, and when it is a substring, the named entity is deduplicated from the standard named entity set. The named entity substring removal operation is performed after clustering to obtain the standard named entity set to prevent the removal of the correct named entity. For example, in some cases, due to OCR errors, image noise is recognized as characters, and "CHINA CITIC BANK" is recognized as "CHINA CITICBANKl". If the substring is removed in advance, the real entity "CHINA CITIC BANK" will be removed, and after clustering, the wrong entities such as "CHINACITIC BANKl" will definitely be a minority and will not be selected as standard named entities for the next step, so there will be no problem of mistakenly deleting standard entities.
[0098] In a specific implementation of the present disclosure, the step S41 may further include step S411 and step S412.
[0099] Step S411, based on the named entity clustering result, traverse and calculate the sum of similarities between each named entity and other named entities in the cluster, to obtain the sum of similarities of each named entity in each cluster;
[0100] Step S412: Arrange the sum of similarities of each of the named entities in each cluster in descending order, and take the named entity with the highest sum of similarities as the standard named entity of each cluster.
[0101] In this embodiment, the voting method is used to remove duplicates in each cluster. Specifically, the sum of the similarities between each named entity and other named entities in the cluster is calculated, which is equivalent to the number of votes cast by other named entities for the corresponding similarity. Then, the named entities are sorted in descending order according to the cumulative number of votes, and the named entity with the highest number of votes is taken as the standard named entity of the cluster. When the named entity with the highest number of votes is not unique, the following alternatives can be used:
[0102] (1) Human judgment;
[0103] (2) According to the actual scenario, select the named entity with the longest or shortest string length;
[0104] (3) Select the named entities that appear more frequently in previous transactions.
[0105] In a specific implementation of the present disclosure, the step S411 may further include step S4111 and step S4112.
[0106] Step S4111, calculating the edit distance between one of the named entities and another of the named entities in each cluster, and calculating the sum of the string lengths of one of the named entities and another of the named entities in each cluster;
[0107] Step S4112: Obtain the similarity between one of the named entities and another of the named entities based on the sum of the edit distance and the character string length.
[0108] The present embodiment is specifically as follows:
[0109] Count any two named entities e i and e j The edit distance between ij , edit distance refers to a string string i Convert to another string j The minimum number of edits, including insert, delete and update. The edit distance calculation formula is as follows:
[0110] edit_dis ij =w d ×delete+w i ×insert+w u ×update (1)
[0111] In the formula, wd is the weight of a deletion operation, w i is the weight of an insertion operation, w u is the weight of a replacement operation. In this embodiment, the weights of these three types of operations are all set to 1, insert is the number of insertions, delete is the number of deletions, and update is the number of replacements;
[0112] Count named entities i and e j The sum of the string lengths is calculated as:
[0113] len_sum ij =len(string i )+len(string j ) (2)
[0114] In the formula, len(string i ) is a named entity e i The length of the string, len(string j ) is a named entity e i The length of the string;
[0115] Count named entities i and e j The similarity between them is calculated as:
[0116] sim ij =(len_sum ij -edit_dis ij ) / edit_dis ij (3)
[0117] In this embodiment, replacement and insertion and deletion are regarded as equivalent operations, and the weights of the corresponding operations are all 1. Because common OCR recognition errors are mainly missed recognition, multiple recognition and wrong recognition. Taking CITIC BANK as an example, missed recognition: CITC BANK, multiple recognition: CCITIC BANK, wrong recognition: CIT1C BANK, these three types of errors have the same impact on the named entity clustering results, so the weights of the edit distance are consistent. Similarly, if in other scenarios, you need to focus on a certain type of error, you can increase the editing weight of this type of error, thereby reducing the similarity between the erroneous string and the true string.
[0118] The similarity calculation in step a) of step (3) above may also be performed using the method in this embodiment.
[0119] In a specific implementation of the present disclosure, the step S5 may further include step S51 and step S52.
[0120] Step S51, obtaining denoising rules input by a staff member;
[0121] Step S52: Perform denoising processing on each of the standard named entities in the standard named entity set according to the denoising rule to obtain a named entity extraction result of the business.
[0122] In this embodiment, the denoising rules can be customized according to the needs, for example, the denoising process can be performed by setting the character length threshold and case discrimination denoising rules. The flexibility and applicability of this embodiment can be improved by customizing the denoising rules.
[0123] Example 2
[0124] like Figure 2 As shown, this embodiment provides a device for extracting business named entities, which includes an acquisition module 701 , a first calculation module 702 , a second calculation module 703 , a third calculation module 704 and a denoising module 705 .
[0125] An acquisition module 701 is used to acquire an image set, where the image set is a set of images containing business information generated when processing the business;
[0126] A first calculation module 702, configured to obtain a named entity corresponding to each image based on each image in the image set, and to group the named entities to obtain a named entity set;
[0127] A second calculation module 703 is used to process the named entity set based on a clustering algorithm to obtain a named entity clustering result;
[0128] A third calculation module 704 is used to process the named entity clustering result based on a voting method to obtain a standard named entity set;
[0129] The denoising module 705 is used to perform denoising processing on the standard named entity set to obtain the named entity extraction result of the business.
[0130] In this embodiment, multiple images in a business transaction can be used as input. The image information in the same business transaction can verify the reliability of the extracted named entity information. The content of a single image is easily affected by noise. The method in this embodiment can improve the accuracy of the named entity extraction result of the final business.
[0131] In a specific implementation of the present disclosure, the first calculation module 702 further includes a first calculation unit 7021 and an extraction unit 7022 .
[0132] A first calculation unit 7021, configured to obtain an OCR text of each image based on each of the images;
[0133] The extraction unit 7022 is used to extract named entities from the OCR text of each image, and to group the extracted named entities to obtain the named entity group.
[0134] In a specific implementation of the present disclosure, the third computing module 704 further includes a deduplication unit 7041 , a collection unit 7042 and an analysis unit 7043 .
[0135] A deduplication unit 7041 is used to perform deduplication processing on each cluster by using a voting method based on the named entity clustering cluster division result to obtain a standard named entity corresponding to each cluster;
[0136] A collection unit 7042, used for collecting the standard named entities corresponding to each cluster to obtain a first standard named entity collection;
[0137] The analysis unit 7043 is used to analyze the first standard named entity set, wherein, if one of the standard named entities in the first standard named entity set is a substring of another standard named entity, then the one standard named entity is removed from the first standard named entity set to obtain the standard named entity set.
[0138] In a specific implementation of the present disclosure, the deduplication unit 7041 further includes a second calculation unit 70411 and a sorting unit 70412 .
[0139] The second calculation unit 70411 is used to traverse and calculate the sum of similarities between each named entity and other named entities in the cluster based on the named entity clustering result, so as to obtain the sum of similarities of each named entity in each cluster;
[0140] The sorting unit 70412 is used to arrange the sum of similarities of each of the named entities in each cluster in descending order, and take the named entity with the highest sum of similarities as the standard named entity of each cluster.
[0141] In a specific implementation of the present disclosure, the second computing unit 70411 further includes a third computing unit 704111 and a fourth computing unit 704112.
[0142] The third calculating unit 704111 is used to calculate the edit distance between one of the named entities and another of the named entities in each cluster, and calculate the sum of the string lengths of one of the named entities and another of the named entities in each cluster;
[0143] The fourth calculation unit 704112 is used to obtain the similarity between one of the named entities and another of the named entities based on the sum of the edit distance and the string length.
[0144] In a specific implementation of the present disclosure, the denoising module 705 further includes an acquisition unit 7051 and a denoising unit 7052 .
[0145] An acquisition unit 7051 is used to acquire a denoising rule input by a staff member;
[0146] The denoising unit 7052 is used to perform denoising processing on each of the standard named entities in the standard named entity set according to the denoising rule to obtain a named entity extraction result of the business.
[0147] It should be noted that, regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment of the method, and will not be elaborated here.
[0148] Example 3
[0149] Corresponding to the above method embodiment, the embodiment of the present disclosure further provides a device for extracting business named entities. The device for extracting business named entities described below and the method for extracting business named entities described above can refer to each other.
[0150] Figure 3 FIG. 8 is a block diagram of a device 800 for extracting business named entities according to an exemplary embodiment. Figure 3 As shown, the service naming entity extraction device 800 may include: a processor 801 and a memory 802. The service naming entity extraction device 800 may also include one or more of a multimedia component 803, an input / output (I / O) interface 804, and a communication component 805.
[0151] The processor 801 is used to control the overall operation of the business naming entity extraction device 800 to complete all or part of the steps in the above-mentioned business naming entity extraction method. The memory 802 is used to store various types of data to support the operation of the business naming entity extraction device 800, and these data may include, for example, instructions for any application or method operating on the business naming entity extraction device 800, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory 802 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, disk or optical disk. The multimedia component 803 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 802 or sent via the communication component 805. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 804 provides an interface between the processor 801 and other interface modules, and the above-mentioned other interface modules can be keyboards, mice, buttons, etc. These buttons can be virtual buttons or physical buttons. The communication component 805 is used for wired or wireless communication between the extraction device 800 of the business naming entity and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G or 4G, or a combination of one or more of them, so the corresponding communication component 805 can include: Wi-Fi module, Bluetooth module, NFC module.
[0152] In an exemplary embodiment, the business naming entity extraction device 800 can be implemented by one or more application specific integrated circuits (Application Specific Integrated Circuit, referred to as ASIC), digital signal processors (Digital Signal Processor, referred to as DSP), digital signal processing devices (Digital Signal Processing Device, referred to as DSPD), programmable logic devices (Programmable Logic Device, referred to as PLD), field programmable gate arrays (Field Programmable Gate Array, referred to as FPGA), controllers, microcontrollers, microprocessors or other electronic components, and is used to execute the above-mentioned business naming entity extraction method.
[0153] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned method for extracting business named entities are implemented. For example, the computer-readable storage medium may be the above-mentioned memory 802 including program instructions, and the above-mentioned program instructions may be executed by the processor 801 of the business named entity extraction device 800 to complete the above-mentioned method for extracting business named entities.
[0154] Example 4
[0155] Corresponding to the above method embodiment, the embodiment of the present disclosure further provides a readable storage medium. The readable storage medium described below and the method for extracting business named entities described above can be referred to each other in correspondence.
[0156] A readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method for extracting business named entities in the above method embodiment.
[0157] The readable storage medium may specifically be a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or other readable storage medium that can store program codes.
[0158] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A method for extracting business named entities, characterized in that: include: Acquire an image set, where the image set is a set of images containing business information generated when processing the business; Based on each image in the image set, obtaining a named entity corresponding to each image, and grouping the named entities to obtain a named entity set; Processing the named entity set based on a clustering algorithm to obtain a named entity clustering cluster division result; Processing the named entity clustering results based on a voting method to obtain a standard named entity set; Performing denoising processing on the standard named entity set to obtain a named entity extraction result of the business; The voting method is used to process the named entity clustering result to obtain a standard named entity set, including: Based on the named entity clustering result, each cluster is deduplicated using a voting method to obtain a standard named entity corresponding to each cluster; The standard named entities corresponding to each cluster are collected to obtain a first standard named entity set; Analyzing the first standard named entity set, wherein if one of the standard named entities in the first standard named entity set is a substring of another standard named entity, removing the one of the standard named entities from the first standard named entity set to obtain the standard named entity set; Based on the named entity clustering cluster division result, each cluster is deduplicated by using a voting method to obtain a standard named entity corresponding to each cluster, including: Based on the named entity clustering result, traverse and calculate the sum of similarities between each named entity and other named entities in the cluster to obtain the sum of similarities of each named entity in each cluster; Arrange the sum of similarities of each named entity in each cluster in descending order, and take the named entity with the highest sum of similarities as the standard named entity of each cluster; The step of traversing and calculating the sum of similarities between each named entity and other named entities in the cluster based on the named entity clustering result to obtain the sum of similarities between each named entity in each cluster includes: Calculating the edit distance between one of the named entities and another of the named entities in each cluster, and calculating the sum of the string lengths of one of the named entities and another of the named entities in each cluster; The similarity between one of the named entities and another of the named entities is obtained based on the sum of the edit distance and the character string length.
2. The method for extracting business named entities according to claim 1, characterized in that: The step of obtaining a named entity corresponding to each image based on each image in the image set, and aggregating the named entities to obtain a named entity set includes: Obtaining OCR text of each image based on each of the images; Named entity extraction is performed on the OCR text of each image, and the extracted named entities are grouped to obtain the named entity set.
3. The method for extracting business named entities according to claim 1, characterized in that: The denoising process is performed on the standard named entity set to obtain the named entity extraction result of the business, including: Obtain denoising rules input by staff; De-noising is performed on each of the standard named entities in the standard named entity set according to the de-noising rule to obtain a named entity extraction result of the business.
4. A device for extracting business named entities, characterized in that: include: An acquisition module, used to acquire an image set, wherein the image set is a set of images containing business information generated when processing the business; A first calculation module, configured to obtain a named entity corresponding to each image based on each image in the image set, and to group the named entities to obtain a named entity set; A second computing module is used to process the named entity set based on a clustering algorithm to obtain a named entity clustering cluster division result; A third calculation module is used to process the named entity clustering result based on a voting method to obtain a standard named entity set; A denoising module, used for performing denoising processing on the standard named entity set to obtain a named entity extraction result of the business; The third computing module comprises: A deduplication unit, configured to perform deduplication processing on each cluster by using a voting method based on the named entity clustering cluster division result, so as to obtain a standard named entity corresponding to each cluster; A collection unit, used for collecting the standard named entities corresponding to each cluster to obtain a first standard named entity collection; An analyzing unit, configured to analyze the first standard named entity set, wherein if one of the standard named entities in the first standard named entity set is a substring of another standard named entity, then the one of the standard named entities is removed from the first standard named entity set to obtain the standard named entity set; The deduplication unit comprises: A second calculation unit is used to traverse and calculate the sum of similarities between each of the named entities and other named entities in the cluster based on the named entity clustering result, so as to obtain the sum of similarities of each of the named entities in each cluster; A sorting unit, used for arranging the sum of similarities of each of the named entities in each cluster in descending order, and taking the named entity with the highest sum of similarities as the standard named entity of each cluster; The second computing unit comprises: A third calculation unit is used to calculate the edit distance between one of the named entities and another of the named entities in each cluster, and calculate the sum of the string lengths of one of the named entities and another of the named entities in each cluster; The fourth calculation unit is used to obtain the similarity between one of the named entities and another of the named entities based on the sum of the edit distance and the character string length.
5. The device for extracting business named entities according to claim 4, characterized in that: The first computing module comprises: A first computing unit, configured to obtain an OCR text of each image based on each of the images; The extraction unit is used to extract named entities from the OCR text of each image, and to group the extracted named entities to obtain the named entity group.
6. The device for extracting business named entities according to claim 4, characterized in that: The denoising module comprises: An acquisition unit, used for acquiring denoising rules input by a staff member; A denoising unit is used to perform denoising processing on each of the standard named entities in the standard named entity set according to the denoising rule to obtain a named entity extraction result of the business.
7. A device for extracting business named entities, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the method for extracting business named entities as claimed in any one of claims 1 to 3 when executing the computer program.
8. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for extracting business named entities according to any one of claims 1 to 3 are implemented.
Citation Information
Patent Citations
Method and equipment for discovering theme key words in picture
CN103544186A