A method and system for extracting job relationships from business cards of people

Abstract

The present invention discloses a method for extracting job relationships for business cards, comprising the following steps: step 1, obtaining a business card image, and preprocessing the business card image; step 2, extracting text from the preprocessed business card image to obtain a text region; step 3, identifying three entities in the text region, the three entities including a name, a work unit, and a position; step 4, correcting the name, work unit, and position identified in step 3; step 5, forming a triple for expressing the job relationship based on the corrected name, work unit, and position and storing it in an electronic business card database. The present invention also discloses a job relationship extraction system for business cards, thereby realizing automatic entry and storage of the job relationship of the business card, and expanding and managing the personal network relationship.

Description

Technical Field

[0001] The present invention relates to the technical field of search and information extraction, and in particular to a method and system for extracting job relationships oriented to business cards of people. Background Art

[0002] Business cards are an important carrier of identity information in today's business interactions and daily life. Their daily use plays a huge role in establishing communication, deepening impressions, and establishing initial business mutual trust. They are also a cost-effective tool for improving personal influence and increasing the possibility of cooperation. In the early days, people often converted the content on business cards into electronic information by manual entry and entered it into digital storage devices for preservation and management. However, on the one hand, this is inefficient and powerless when there is a large amount of data to be processed; on the other hand, it is costly. A simple entry work requires people who can use computers to do repetitive work, and later it takes a lot of time and energy to manage and maintain the database, and it is often inconvenient to connect with other people's databases. With the increasing frequency of communication today, the demand for business card entry is increasing. It has become possible and urgent to realize the automatic entry and storage of business cards through technical means.

[0003] In the existing processing methods for business cards, usually only the contact information such as mobile phone numbers and email addresses are accessed. In order to organize and manage personal connections, the position relationship in the business card is very important. The position relationship is expressed by the triple of <name, work unit, position>. The existing methods do not solve the following problems:

[0004] (1) Identify three entities: name, work unit, and position from the text extraction results;

[0005] (2) Correct the misrecognized text according to the characteristics of the entity;

[0006] (3) For multiple work units and positions, match them and generate the correct <name, work unit, position> triple. Summary of the invention

[0007] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide a method and system for extracting job relationships for personal business cards. The present invention extracts text from pictures, automatically identifies three entities: name, work unit, and position, and corrects the erroneous entities extracted according to the characteristics of different entities. Finally, a method for matching and generating several job relationships is performed, thereby realizing automatic entry and storage of job relationships on business cards and expanding and managing personal connections.

[0008] The present invention adopts the following technical solutions to solve the above technical problems:

[0009] A method for extracting job relationships based on business cards of people proposed in the present invention comprises the following steps:

[0010] Step 1: Obtain a business card image and pre-process the business card image;

[0011] Step 2: extract the text in the preprocessed business card image to obtain a text area;

[0012] Step 3: Identify three entities in the text area, including name, work unit and position;

[0013] Step 4, modify the name, work unit and position identified in step 3;

[0014] Step 5: Based on the corrected name, work unit and position, several triples are formed to express the employment relationship and stored in the electronic business card database, each triple being <name, work unit, position>.

[0015] As a further optimization scheme of the method for extracting job relationships for business cards of a person described in the present invention, the way of obtaining the business card picture of the person in step 1 is: shooting, crawling or user-provided;

[0016] Preprocessing includes:

[0017] If the business card image contains multiple business cards, it is first segmented into individual business cards, and then the individual business cards are binarized, noise smoothed, tilt angle detected and corrected.

[0018] As a further optimization scheme of the method for extracting job relationships for business cards of people described in the present invention, the extraction in step 2 includes text detection and text recognition.

[0019] As a further optimization scheme of the method for extracting job relationships for business cards of a person described in the present invention,

[0020] After step 1, it also includes automatically generating a picture training test set from the pre-processed business card pictures;

[0021] In step 2, the text is extracted using business card images in the automatically generated image training test set. The method for automatically generating the training test set includes generating images of Chinese characters in multiple fonts with different noises, and generating multiple test samples by automatically adjusting the angle of the business card images.

[0022] As a further optimization scheme of the method for extracting job relationships for business cards described in the present invention, in step 3, three entities of name, work unit and position are identified based on the method of named entity recognition. When more than two entities are contained in the same text area, they are divided into single entities using Chinese lexical tools.

[0023] As a further optimization scheme of the method for extracting the position relationship for the business card of a person described in the present invention, in step 4, the correction method is as follows:

[0024] Step ①, for the recognized name, if there is a corresponding pinyin of the name in the name card picture, use the Chinese character pinyin database to obtain the Chinese character with the same pinyin and the closest glyph for correction; if there is no pinyin, use the glyph similarity measurement to select the Chinese character with the closest glyph for correction;

[0025] Step ②: for the identified work unit, determine whether it is a logo based on its position and font. If it is a logo, call the logo recognition algorithm to identify and correct it; if it is not a logo but contains the English, pinyin or address information of the work unit, use the English, pinyin or address information as input, call the search engine interface to search for the correct name of the work unit for correction; if it does not contain the above information, first use the language model to obtain the word, and then use the glyph similarity measurement to select the font closest to the Chinese character for correction;

[0026] Step ③: For the identified positions, select the position name with the smallest editing distance according to the position dictionary for correction; if the distances of the position names in the dictionary are all greater than the preset threshold, input the corrected work unit name and the position to be corrected into the language model together to obtain the most likely character, and then use the glyph similarity measurement to select the glyph closest to the Chinese character for correction.

[0027] As a further optimization scheme of the method for extracting job relationships for business cards of a person described in the present invention, in step 5, if there are multiple work units and positions, the work units and positions are paired according to the proximity of their positions; if a certain position has no adjacent work unit in terms of position, the identified logo is used as the work unit corresponding to the position.

[0028] A job relationship extraction system for business cards, including

[0029] The image training test set unit is used to store business card images of people with different noises in multiple fonts containing Chinese characters, and to automatically adjust the angle of the business card images to generate multiple test sample business card images;

[0030] A text knowledge base unit is used to store a Chinese character pinyin library, a stroke order library, and a dictionary of job titles and unit names;

[0031] A text extraction unit is used to extract text from a business card image and obtain a text extraction result, which is output to an entity recognition unit. The text extraction result includes a text area.

[0032] The entity recognition unit is used to recognize the three entities of name, work unit and position in the text extraction result; it also includes using Chinese lexical tools to divide the same text area into single entities when two or more entities are contained in the same text area;

[0033] An entity correction unit, used to correct the parts of the recognized names, work units and positions whose confidence is lower than a preset value;

[0034] The position relationship generation unit is used to generate a number of <name, work unit, position> triplets and store them in the database;

[0035] The entity correction unit includes a person name correction subunit, a work unit correction subunit and a position correction subunit:

[0036] The name correction subunit is used to correct the recognized name by using the Chinese character pinyin database to obtain the Chinese character with the same pinyin and the closest glyph when the corresponding name pinyin exists in the name card picture; if the pinyin does not exist, the Chinese character with the closest glyph is selected for correction by using the glyph similarity measurement;

[0037] The work unit correction subunit determines whether the identified work unit is a logo based on its position and font. If it is a logo, the logo recognition algorithm is called to identify and correct it; if it is not a logo but contains the English, pinyin or address information of the work unit, the English, pinyin or address information is used as input, and the search engine interface is called to search for the correct name of the work unit for correction; if it does not contain the above information, the language model is first used to obtain the word, and then the word shape similarity measurement is used to select the font closest to the Chinese character for correction;

[0038] The position correction subunit is used to select the position name with the smallest editing distance for correction according to the position dictionary library for the identified position; if the distances of the position names in the dictionary are all greater than a preset threshold, the corrected work unit name and the position to be corrected are input into the language model together to obtain the most likely character, and then the character with the closest font to the Chinese character is selected for correction using the glyph similarity measurement.

[0039] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:

[0040] The method and system of the present invention can automatically realize the following functions:

[0041] (1) Identify three entities: name, work unit, and position from the text extraction results;

[0042] (2) Correct the misrecognized text according to the characteristics of the entity;

[0043] (3) Match multiple work units and positions to generate several correct <name, work unit, position> triplets;

[0044] This improves the accuracy of extracting job relationships, enables automatic entry and storage of job relationships in business cards, and increases the viewing rate and dissemination of electronic business cards. Based on the extracted job relationships, personal connections can be managed and expanded. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a flow chart of a method for extracting job relationships based on person business cards;

[0046] Figure 2 It is a structure diagram of the position relationship extraction system for person business cards;

[0047] Figure 3 It is a structural diagram of the text extraction unit;

[0048] Figure 4 It is a structural diagram of the entity correction unit. DETAILED DESCRIPTION

[0049] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings:

[0050] like Figure 1 As shown, a method for extracting job relationships based on business cards of people includes the following steps:

[0051] Step 1. Obtain a business card image and pre-process the image;

[0052] Step 2. Extract the text in the picture;

[0053] Step 3. For the extracted text area, identify the three entities of name, work unit and position;

[0054] Step 4. For the person's name, work unit and position identified in step 3, if the text recognition confidence is lower than the threshold, make corrections;

[0055] Step 5. Form several triplets of <name, work unit, position> and store them in the database.

[0056] The business card image in step 1 can be obtained by photographing, obtained from the Internet using a crawler, or provided by the user. The preprocessing steps for the image include:

[0057] (1) If the image contains multiple business cards, first segment them into individual business cards;

[0058] (2) Perform image binarization, noise smoothing, tilt angle detection and correction, etc.

[0059] The extraction of text from the image in step 2 includes two steps: text detection and text recognition. Text detection can use a detection model based on image segmentation such as DB to determine whether each pixel belongs to a text target and its connection with surrounding pixels, and then integrate the adjacent pixel results into a text box; the text recognition algorithm can use a CRNN recognition model, including a convolutional layer CNN, a recurrent layer RNN, and a transcription layer CTC.

[0060] Before step 2, the method further includes the step of automatically generating a picture training test set to train the text extraction method. The method of automatically generating a training test set includes generating pictures of 3,000 commonly used Chinese characters in common fonts including Song, Kai, Li and Hei, etc., with different noises, and automatically adjusting the angle of business card pictures to generate multiple test samples.

[0061] In step 3, the named entity recognition method is used to identify the three entities of name, work unit and position. When two or more entities are included in the same area, for example, "CEO of Fujian Moko Consulting Co., Ltd." includes two entities, "work unit" and "position", the Chinese lexical tool is used to divide it into single entities. The lexical tool can be Baidu LAC Chinese lexical analysis tool. The named entity recognition method can use a three-layer model consisting of an embedding layer, a BiLSTM layer, and a decoding CRF layer.

[0062] In step 4, the correction method is as follows:

[0063] (1) For the recognized name, if the corresponding pinyin of the name exists in the business card, the Chinese character with the same pinyin and the closest glyph is obtained from the Chinese character pinyin database for correction; if the pinyin does not exist, the glyph similarity measurement is used to select the Chinese character with the closest glyph for correction. The glyph similarity can be calculated based on the edit distance of IDS (Ideographic Description Sequence).

[0064] (2) For the identified work unit, determine whether it is a logo based on its position, font and other information. If it is a logo, call the logo recognition algorithm for identification and correction; if it is not a commonly used logo, but contains the English, pinyin or address information of the work unit, use the English, pinyin or address information as input, call the search engine interface to search for the correct name of the work unit for correction; if it does not contain the above information, first use the language model to get the most likely word, and then use the glyph similarity measurement to select the glyph closest to the Chinese character for correction. The language model can use the BERT model, and the glyph similarity can be calculated based on IDS.

[0065] (3) For the identified positions, select the position name with the smallest edit distance according to the position dictionary for correction; if the distances of the position names in the dictionary are all greater than the threshold, input the corrected work unit name and the position to be corrected into the language model together to obtain the most likely character, and then use the glyph similarity metric to select the glyph closest to the Chinese character for correction among the most likely characters. The language model can use the BERT model, and the glyph similarity can be calculated based on IDS.

[0066] In step 5, there may be multiple work units and positions, and the work units and positions are paired according to the proximity of the positions. If there is no work unit adjacent to a position, the recognized logo is used as the work unit corresponding to the position.

[0067] like Figure 2 As shown in the figure, a position relationship extraction system for business cards includes the following parts:

[0068] Image training test set unit: including images of 3,000 commonly used Chinese characters in common fonts, including Songti, Kaiti, Lishu and Heiti, with different noises, as well as image data such as business card images that automatically adjust the angle to generate multiple test samples;

[0069] Text knowledge base unit: including Chinese character pinyin database, stroke order database, and dictionaries such as job titles and unit names;

[0070] Text extraction unit: realizes the extraction of text from the picture of the business card;

[0071] Entity recognition unit: It realizes the recognition of three entities in the text extraction results, namely, names, work units and positions; it also includes the use of Chinese lexical tools to divide two or more entities into individual entities when the same area contains two or more entities. The lexical tool can use Baidu LAC Chinese lexical analysis tool. Its named entity recognition method can use a three-layer model consisting of an embedding layer, a BiLSTM layer, and a decoding CRF layer.

[0072] Entity correction unit: It is used to correct the low-reliability parts of the recognized names, work units and job positions.

[0073] Position relationship generation unit: generates several triplets of <name, work unit, position> and stores them in the database.

[0074] like Figure 3 As shown, the text extraction unit includes a detection subunit and a recognition subunit, which respectively realize text detection and text recognition; the text detection can use a detection model based on image segmentation such as DB to determine whether each pixel belongs to a text target and its connection with surrounding pixels, and then integrate the adjacent pixel results into a text box; the text recognition algorithm can use a CRNN recognition model, including a convolutional layer CNN, a recurrent layer RNN and a transcription layer CTC.

[0075] like Figure 4 As shown, the entity correction unit includes a name correction subunit, a work unit correction subunit and a position correction subunit:

[0076] (1) For the recognized name, if the corresponding pinyin of the name exists in the business card, the name correction subunit uses the Chinese character pinyin database to obtain the Chinese character with the same pinyin and the closest glyph to correct it; if the pinyin does not exist, the glyph similarity measurement is used to select the Chinese character with the closest glyph to correct it; the glyph similarity can be calculated based on the edit distance of IDS (Ideographic Description Sequence).

[0077] (2) The work unit correction subunit determines whether the identified work unit is a logo based on its position, font and other information. If it is a logo, the logo recognition algorithm is called for identification and correction. If it is not a commonly used logo, but it contains the English, pinyin or address information of the work unit, the English, pinyin or address information is used as input to call the search engine interface to search for the correct name of the work unit for correction. If it does not contain the above information, the language model is first used to obtain the most likely word, and then the glyph similarity measurement is used to select the glyph closest to the Chinese character for correction. The language model can use the BERT model, and the glyph similarity can be calculated based on IDS.

[0078] (3) The position correction subunit selects the position name with the smallest edit distance for correction according to the position dictionary for the identified position. If the distances of the position names in the dictionary are all greater than the threshold, the corrected work unit name and the position to be corrected are input into the language model together to obtain the most likely character, and then the glyph similarity measurement is used to select the glyph closest to the Chinese character for correction among the most likely characters. The language model can use the BERT model, and the glyph similarity can be calculated based on IDS.

[0079] The above description is only a specific implementation of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a technician familiar with the technical field within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention.

Claims

1. A method for extracting job relationships based on business cards, characterized in that: The following steps are involved: Step 1: Obtain a business card image and pre-process the business card image; Step 2: extract the text in the preprocessed business card image to obtain a text area; Step 3: Identify three entities in the text area, including name, work unit and position; Step 4, modify the name, work unit and position identified in step 3; Step 5: Based on the corrected name, work unit and position, several triples are formed to express the employment relationship and stored in the electronic business card database, each triple being <name, work unit, position>; In step 4, the correction method is as follows: Step ①, for the recognized name, if there is a corresponding pinyin of the name in the business card image, use the Chinese character pinyin database to obtain the Chinese character with the same pinyin and the closest glyph to make corrections; In the absence of pinyin, the font closest to the Chinese character is selected for correction using the glyph similarity metric; Step ②: for the identified work unit, determine whether it is a logo based on its position and font. If it is a logo, call the logo recognition algorithm to identify and correct it; if it is not a logo but contains the English, pinyin or address information of the work unit, use the English, pinyin or address information as input, call the search engine interface to search for the correct name of the work unit for correction; if it does not contain the above information, first use the language model to obtain the word, and then use the glyph similarity measurement to select the font closest to the Chinese character for correction; Step ③: For the identified positions, select the position name with the smallest editing distance according to the position dictionary for correction; if the distances of the position names in the dictionary are all greater than the preset threshold, input the corrected work unit name and the position to be corrected into the language model together to obtain the most likely character, and then use the glyph similarity measurement to select the glyph closest to the Chinese character for correction.

2. The method for extracting job relationships based on business cards of a person according to claim 1, characterized in that: The way to obtain the business card image of the person in step 1 is: shooting, crawling or user-provided; Preprocessing includes: If the business card image contains multiple business cards, it is first segmented into individual business cards, and then the individual business cards are binarized, noise smoothed, tilt angle detected and corrected.

3. The method for extracting job relationships based on business cards according to claim 1, characterized in that: The extraction in step 2 includes text detection and text recognition.

4. The method for extracting job relationships based on business cards according to claim 1, characterized in that: After step 1, it also includes automatically generating a picture training test set from the pre-processed business card pictures; In step 2, the text is extracted using business card images in the automatically generated image training test set. The method for automatically generating the training test set includes generating images of Chinese characters in multiple fonts with different noises, and generating multiple test samples by automatically adjusting the angle of the business card images.

5. The method for extracting job relationships based on business cards according to claim 1, characterized in that: In step 3, three entities, namely, person's name, work unit and position, are identified based on the named entity recognition method. When the same text region contains more than two entities, they are divided into individual entities using Chinese lexical tools.

6. The method for extracting job relationships based on business cards according to claim 1, characterized in that: In step 5, if there are multiple work units and positions, the work units and positions are paired according to the proximity of their locations; if there is no work unit adjacent to a position, the identified logo is used as the work unit corresponding to the position.

7. A system for extracting job relationships from business cards, characterized in that: include The image training test set unit is used to store business card images of people with different noises in multiple fonts containing Chinese characters, and to automatically adjust the angle of the business card images to generate multiple test sample business card images; A text knowledge base unit is used to store a Chinese character pinyin library, a stroke order library, and a dictionary of job titles and unit names; A text extraction unit is used to extract text from a business card image and obtain a text extraction result, which is output to an entity recognition unit. The text extraction result includes a text area. The entity recognition unit is used to recognize the three entities of name, work unit and position in the text extraction result; it also includes using Chinese lexical tools to divide the same text area into single entities when two or more entities are contained in the same text area; An entity correction unit, used to correct the parts of the recognized names, work units and positions whose confidence is lower than a preset value; The position relationship generation unit is used to generate a number of <name, work unit, position> triplets and store them in the database; The entity correction unit includes a person name correction subunit, a work unit correction subunit and a position correction subunit: The name correction subunit is used to correct the recognized name by using the Chinese character pinyin database to obtain the Chinese character with the same pinyin and the closest glyph when the corresponding name pinyin exists in the name card picture; if the pinyin does not exist, the Chinese character with the closest glyph is selected for correction by using the glyph similarity measurement; The work unit correction subunit determines whether the identified work unit is a logo based on its position and font. If it is a logo, the logo recognition algorithm is called to identify and correct it; if it is not a logo but contains the English, pinyin or address information of the work unit, the English, pinyin or address information is used as input, and the search engine interface is called to search for the correct name of the work unit for correction; if it does not contain the above information, the language model is first used to obtain the word, and then the word shape similarity measurement is used to select the font closest to the Chinese character for correction; The position correction subunit is used to select the position name with the smallest editing distance for correction according to the position dictionary library for the identified position; if the distances of the position names in the dictionary are all greater than a preset threshold, the corrected work unit name and the position to be corrected are input into the language model together to obtain the most likely character, and then the character with the closest font to the Chinese character is selected for correction using the glyph similarity measurement.

Images