Resume information extraction method and device, electronic equipment and storage medium

Abstract

The present application provides a resume information extraction method, device, electronic equipment and storage medium, belongs to image processing technical field, the resume information extraction method includes: the text line detection and text recognition are carried out to target resume image, obtain the position information of each text line in target resume image and the text recognition result corresponding to each text line; based on the position information and text recognition result of each text line, each text line in target resume image is sorted, and the text line content after sorting is obtained; the text line content after sorting is encoded and decoded, and the text line structured information of target resume image is obtained. The present application can break through various restrictions on the resume by obtaining the text line structured information of the resume image, accurately extract useful information from resumes of various characteristics and various types, effectively improve the resume screening efficiency, and reduce the screening error rate.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, electronic device, and storage medium for extracting resume information. Background Technology

[0002] When job hunting, applicants typically upload their resumes to recruiting companies' systems or websites by scanning or photographing them. Since a resume is the interviewer's first impression, most applicants personalize their resumes to highlight their value or strengths, resulting in no format restrictions. During peak job-hunting season, staff must extract useful information from a large volume of diverse resumes and select those that meet interview requirements. This process is labor-intensive, inefficient, and prone to errors. Therefore, finding a solution to extract useful information from these diverse resumes is crucial. Summary of the Invention

[0003] This invention provides a method, apparatus, electronic device, and storage medium for extracting resume information, in order to solve the problem of how to extract useful information from resumes of various types and characteristics.

[0004] This invention provides a method for extracting resume information, comprising:

[0005] The target resume image is subjected to text line detection and text recognition to obtain the position information of each text line in the target resume image and the text recognition result corresponding to each text line.

[0006] Based on the position information and text recognition results of each text line, the text lines in the target resume image are sorted to obtain the sorted text line content;

[0007] The sorted text lines are encoded and decoded to obtain the text line structured information of the target resume image.

[0008] According to a resume information extraction method provided by the present invention, the step of performing text line detection and text recognition on a target resume image to obtain the position information of each text line in the target resume image and the text recognition result corresponding to each text line includes:

[0009] Perform text line detection on the target resume image to obtain the position information of each text line in the target resume image;

[0010] Based on the position information of each text line, each text line in the target resume image is corrected to obtain a text line corrected image;

[0011] Text recognition is performed on the text line correction image to obtain the text line recognition result corresponding to each text line.

[0012] According to a resume information extraction method provided by the present invention, the step of correcting each text line in the target resume image based on the position information of each text line to obtain a text line corrected image includes:

[0013] Based on the position information of each text line, the line image corresponding to each text line in the target resume image is obtained by using the bounding rectangle method.

[0014] Based on the line image corresponding to each text line, the rotated image corresponding to each text line is obtained using the minimum bounding rectangle method.

[0015] Based on the rotated image corresponding to each text line, calculate the rotation angle of each text line;

[0016] Based on the rotation angle of each text line, an affine transformation is performed on each text line to obtain the line image corresponding to each text line after rotation correction;

[0017] The interference information in the line image corresponding to each line of text after rotation correction is removed by using a mask image to obtain the text line correction image.

[0018] According to a resume information extraction method provided by the present invention, the step of sorting the text lines in the target resume image based on the position information and text recognition results of each text line to obtain the sorted text line content includes:

[0019] Based on the position information of each text line, the height directional overlap between semantic blocks in the target resume image is determined;

[0020] Based on the height directional overlap between semantic blocks in the target resume image, the semantic blocks are sorted.

[0021] The text lines within each semantic block are sorted to obtain the sorted text line content.

[0022] According to a resume information extraction method provided by the present invention, the step of encoding and decoding the sorted text line content to obtain the text line structured information of the target resume image includes:

[0023] Based on the sorted text line content, text features are obtained; based on the position information of each text line and the target resume image, layout features corresponding to each text line are obtained; and based on the target resume image, image features are obtained.

[0024] The text features, layout features, and image features are fused to obtain the fused text line features;

[0025] Hierarchical relationship reasoning is performed on the fused text line features to obtain the structured relationship of the text lines in the target resume image.

[0026] According to a resume information extraction method provided by the present invention, the step of performing hierarchical relationship reasoning on the fused text line features to obtain the text line structured relationship of the target resume image includes:

[0027] Based on the fused text line features, the title, parent node, and relationship with the parent node corresponding to each text line are determined.

[0028] For each text line, determine the hierarchical information of each text line based on the corresponding title and the relationship between each text line and its parent node;

[0029] Based on the hierarchical information of each text line, the structured relationship of the text lines in the target resume image is obtained.

[0030] According to a resume information extraction method provided by the present invention, the method further includes:

[0031] Based on the text line structure relationship of the target resume image, the semantic block to which each text line belongs is verified to obtain the verification result;

[0032] The semantic block is processed based on the verification result.

[0033] The present invention also provides a resume information extraction device, comprising:

[0034] The text detection and recognition module is used to perform text line detection and text recognition on the target resume image to obtain the position information of each text line in the target resume image and the text recognition result corresponding to each text line.

[0035] The text sorting module is used to sort the text lines in the target resume image based on the position information and text recognition results of each text line, so as to obtain the sorted text line content;

[0036] The encoding / decoding processing module is used to encode and decode the sorted text lines to obtain the text line structured information of the target resume image.

[0037] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the resume information extraction method as described above.

[0038] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the resume information extraction method as described above.

[0039] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the resume information extraction method as described above.

[0040] The resume information extraction method, apparatus, electronic device, and storage medium provided by this invention perform text line detection and text recognition on a target resume image to obtain the position information and text recognition results of each text line; based on the position information and text recognition results of each text line, sort the text lines in the target resume image to obtain the sorted text line content; and perform encoding and decoding processing on the sorted text line content to obtain the structured text line information of the target resume image. This can overcome various limitations on resumes, enabling the extraction of useful information from resumes of various characteristics and types, effectively improving resume screening efficiency and reducing screening error rate. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0042] Figure 1 This is a diagram illustrating the application environment in which the resume information extraction method provided in this embodiment of the invention can be run.

[0043] Figure 2 A flowchart illustrating the resume information extraction method provided by this invention;

[0044] Figure 3 Example image of a target resume provided in an embodiment of the present invention;

[0045] Figure 4 Example diagram of a text line provided in an embodiment of the present invention;

[0046] Figure 5 Example diagram of text line structured information provided in embodiments of the present invention;

[0047] Figure 6 This is a schematic diagram illustrating the process of performing text line detection and text recognition on a target resume image according to an embodiment of the present invention;

[0048] Figure 7 This is a flowchart illustrating the process of correcting each line of text in a target resume image according to an embodiment of the present invention.

[0049] Figure 8This is a schematic diagram of the process for encoding and decoding sorted text lines provided in an embodiment of the present invention;

[0050] Figure 9 A flowchart illustrating hierarchical relationship reasoning provided in an embodiment of the present invention;

[0051] Figure 10 A schematic diagram of the structure of the resume information extraction device provided by the present invention;

[0052] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0054] Most existing resume information extraction solutions directly extract text from electronic resumes, encode the text, and then use models such as convolutional neural networks or recurrent neural networks to learn and extract resume information. Alternatively, they preset relevant attributes such as "education experience," "project experience," and "work experience," and extract specific fields such as "place name," "time," and "responsibility" through different rules to achieve the purpose of information extraction. Still others restrict a uniform format, such as HTML, to extract simple structured information such as name, salary, and work location. Existing resume information extraction solutions either restrict the source of the resume, limit the resume format, or restrict the language to Chinese. For complex scenarios, such as complex sentences, multi-layered nested content, incorrect photo angles, complex background information, or English resumes, they cannot effectively extract resume information and may miss key information.

[0055] To address this, the present invention provides a method, apparatus, electronic device, and storage medium for extracting resume information. By performing text line detection and text recognition on a target resume image, the position information and text recognition results of each text line are obtained. Based on the position information and text recognition results of each text line, the text lines in the target resume image are sorted to obtain the sorted text line content. The sorted text line content is then encoded and decoded to obtain the structured text line information of the target resume image. By obtaining the structured text line information of a resume image, this invention can overcome various limitations on resumes, enabling the extraction of useful information from resumes of diverse characteristics and types, effectively improving resume screening efficiency and reducing screening error rates.

[0056] The resume information extraction method provided by this invention can be applied to, for example... Figure 1 The application environment shown. Figure 1 This diagram illustrates the application environment in which the resume information extraction method provided in this embodiment of the invention can be run. For example... Figure 1 As shown, the application environment includes terminal 110 and server 120. Terminal 110 and server 120 communicate via a network, which can be a wireless communication network or a wired communication network. The number of terminals and servers is unlimited. The wireless communication network can include, but is not limited to, at least one of the following: WIFI (Wireless Fidelity) and Bluetooth. The wired communication network can include, but is not limited to, at least one of the following: wide area network, metropolitan area network, and local area network.

[0057] Terminal 110 includes various handheld devices, in-vehicle devices, wearable devices, computing devices, or other processing devices connected to a wireless modem with wireless or wired communication capabilities, such as mobile phones, tablets, desktop laptops, and smart devices capable of running applications, including the central console of a smart car, smartphones, etc.

[0058] Server 120 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0059] It should be noted that the resume information extraction method in this invention can be implemented directly on the terminal 110, or directly on the server 120, or it can be completed on the server 120 and then sent to the terminal 110 by the server 120.

[0060] Terminal 110 or server 120 performs text line detection and text recognition on the target resume image to obtain the position information and text recognition results of each text line. Based on the position information and text recognition results of each text line, the text lines in the target resume image are sorted to obtain the sorted text line content. The sorted text line content is then encoded and decoded to obtain the structured information of the text lines in the target resume image. This method can overcome the limitations of resume format and extract useful information from resumes of various types and characteristics, effectively improving resume screening efficiency and reducing screening error rate. The following description uses the execution of the resume information extraction method of this invention by a terminal as an example.

[0061] Figure 2 This is a flowchart illustrating the resume information extraction method provided by the present invention. Figure 2 As shown, a method for extracting resume information is provided, which can be applied to... Figure 1 The following steps are used as an example of the terminal in the example: Step 210, Step 220, and Step 230.

[0062] Step 210: Perform text line detection and text recognition on the target resume image to obtain the position information of each text line in the target resume image and the text recognition result corresponding to each text line;

[0063] The target resume image can be obtained by the job seeker uploading their personal resume document to the recruitment company's recruitment system or back-end server by scanning or taking a photo, and then reading the resume image stored in the terminal database or server database.

[0064] The target resume image can also be obtained by scanning or photographing the job seeker's personal resume document, sending the resume image to the recruitment company's recruitment email address, and then retrieving the resume image from the recruitment email address.

[0065] The target resume image can also be obtained by having job seekers send their resume documents directly to the recruitment company's recruitment email address, and then converting the various types of resume documents in the recruitment email address into images.

[0066] Figure 3 An example image of a target resume provided in an embodiment of the present invention. Figure 3 As shown, a target resume image can be divided into multiple text blocks. Figure 3 In this context, A, B, C, D, and E each represent a text block.

[0067] A text block can contain multiple lines of text that have semantic relationships. Figure 4 An example diagram of a text line provided for an embodiment of the present invention. For example... Figure 4 As shown, the personal information text block includes multiple text lines, each of which corresponds to a numerical code.

[0068] The text line detection and text recognition of the target resume image include the following two steps:

[0069] The first step is to perform text line detection to obtain the positional information of the contour points of each text line in the target resume image. Using this contour point positional information, the accurate location of each text line can be determined.

[0070] Second, text recognition is performed to obtain the text recognition results corresponding to each line of text in the target resume image. The content expressed by each line of text can then be determined from the text recognition results.

[0071] By performing text line detection and text recognition on the target resume image, the integrity of the resume information extraction can be guaranteed, and the potential loss of information when extracting text lines in complex scenarios can be avoided from the outset.

[0072] Step 220: Based on the position information and text recognition results of each text line, sort each text line in the target resume image to obtain the sorted text line content;

[0073] In this embodiment, in order to improve the readability of the text recognition results and overcome the ambiguity caused by the reading order, after obtaining the position information and text recognition results of each text line, it is necessary to sort the text lines in the target resume image based on the position information and text recognition results of each text line to obtain the sorted text line content.

[0074] In some embodiments, the position information of each text line and the text recognition result can be input into the semantic block generation model to obtain the semantic block output by the semantic block generation model.

[0075] It should be noted that the semantic block output by the semantic block generation model includes several sorted text lines. A semantic block can be understood as a paragraph composed of several text lines. Therefore, by obtaining the semantic block output by the semantic block generation model, the sorted text line content can be obtained.

[0076] The semantic block generation model is trained based on the positional information samples of each text line, the text recognition result samples, and the corresponding semantic block samples.

[0077] Based on the position information and text recognition results of each text line, this invention sorts each text line in the target resume image to obtain the sorted text line content, which can improve the readability of the text recognition results and enable the extraction of effective information even when the resume has multiple columns or nested elements.

[0078] Step 230: Encode and decode the sorted text lines to obtain the text line structured information of the target resume image.

[0079] The structured information of the text lines includes the hierarchical information of each text line in the target resume image, the category corresponding to the hierarchical information, and the hierarchical relationship between the text lines.

[0080] The hierarchical information and the corresponding categories can be predefined. For example, we can define level 1 as a first-level heading and level 2 as text line content; that is, level 1 corresponds to the category of first-level heading and level 2 corresponds to the category of text line content.

[0081] In some embodiments, hierarchical information and the corresponding categories can be determined or defined based on the resume's reading rules or habits. For example, following general reading habits, text lines with semantic relationships are divided into text blocks. The reading order between text blocks is from top to bottom and from left to right. For each text block, it is divided into text lines according to reading habits. Then, the hierarchical information of each text line and the corresponding category are determined. For example, refer to... Figure 4 It can Figure 4 The most recent job, highest education / degree, and personal information are defined as the first level, and the categories corresponding to these are defined as first-level headings. The job title, company, and industry under "most recent job" are defined as the second level, and the categories corresponding to these are defined as second-level headings.

[0082] In one embodiment, the hierarchical information and the corresponding categories of the hierarchical information are defined according to a conventional resume format, as shown in Table 1.

[0083] Table 1. Hierarchical information and corresponding categories.

[0084]

[0085]

[0086] It should be noted that the hierarchy information and the corresponding categories in Table 1 are merely examples and are not intended to limit the hierarchy and categories in the embodiments of the present invention.

[0087] The hierarchical relationship between lines of text is uncertain and needs to be inferred based on the specific content of the resume.

[0088] The hierarchical relationships between text lines include containment, parallelism, and connection. Among them, connection refers to the ability of two text lines to be combined to form a whole; the relationship between these two text lines is called a connection relationship.

[0089] Figure 5 This is an example diagram illustrating the structured information of text lines provided in an embodiment of the present invention. For example... Figure 5 As shown, work experience is at level 1, work experience is at level 2, and level 3 is the specific job position and job description.

[0090] In this embodiment of the invention, the sorted text lines are encoded and decoded, including: encoding the sorted text lines to obtain encoded features, and then decoding the encoded features to obtain structured information about the text lines. Through the decoding process, the hierarchical relationships between text lines are inferred.

[0091] This invention encodes and decodes the sorted text lines to obtain the structured text line information of the target resume image, which can be used for subsequent retrieval of key information, resume document recovery, or resume recommendation system. This invention does not specifically limit these uses.

[0092] In some embodiments, after obtaining the text line structured information of the target resume image, the resume information extraction method further includes:

[0093] Output the resume information corresponding to the target resume image. The resume information includes at least one of the following: the position information of each text line, the text recognition result corresponding to each text line, the sorted text line content, and the text line structure information.

[0094] It is understandable that the final output will be a series of information, including the position information of each text line, the text recognition result corresponding to each text line, the sorted text line content, and the text line structure information, for subsequent processing.

[0095] After obtaining the sorted text line content, this invention performs encoding and decoding processing on the sorted text line content to obtain the text line structured information of the target resume image. This can solve problems such as multiple columns and multiple layers of nesting in the resume itself. Subsequently, based on the text line structured information, different resume extraction rules or requirements can be set according to the recruitment needs of different recruiting companies, thereby extracting different resume information and breaking through various restrictions on resumes.

[0096] The resume information extraction method provided by this invention obtains the position information and text recognition results of each text line by performing text line detection and text recognition on the target resume image; based on the position information and text recognition results of each text line, the text lines in the target resume image are sorted to obtain the sorted text line content; the sorted text line content is encoded and decoded to obtain the text line structured information of the target resume image. This method can overcome various limitations on resumes and extract useful information from resumes of various types and characteristics, effectively improving resume screening efficiency and reducing screening error rate.

[0097] Figure 6 This is a flowchart illustrating the process of text line detection and text recognition for a target resume image provided in an embodiment of the present invention, such as... Figure 6 As shown, in some embodiments of the present invention, step 210 includes:

[0098] Step 211: Perform text line detection on the target resume image to obtain the position information of each text line in the target resume image;

[0099] Optionally, the target resume image is input into a text line detection model for text line detection, and the output of the text line detection model is obtained to obtain the position information of the contour points of each text line in the target resume image, that is, the position information of each text line. It can be understood that the position information of the contour points of each text line can describe the position of the text line in the target resume image.

[0100] The text line detection model is trained based on resume image samples and the annotation information of the corresponding text lines.

[0101] In some embodiments, the text line detection model may employ the PSENET network model or the DB (Real-time Scene Text Detection with Differentiable Binarization) model.

[0102] PSENet, an instance segmentation network, offers two key advantages. First, as a segmentation-based method, it can locate text of arbitrary shapes. Second, the model proposes a progressive scale-expansion algorithm that successfully identifies adjacent text instances. Using the PSENet network model for text line detection in target resume images effectively extracts text information from them.

[0103] It should be noted that the present invention may also use other models capable of text line detection as text line detection models, and the present invention does not impose specific limitations on this.

[0104] Step 212: Based on the position information of each text line, correct each text line in the target resume image to obtain a text line correction image;

[0105] Since the target resume image may have problems such as incorrect shooting angle or poor lighting, resulting in text lines being tilted or curved, or having shadows, we can first perform text line detection to obtain the position information of each text line. Then, based on the position information of each text line, we can correct the detected text lines to achieve accurate text line correction and obtain a text line corrected image.

[0106] The text recognition results obtained in this way can ensure the completeness and accuracy of resume information extraction.

[0107] Step 213: Perform text recognition on the text line correction image to obtain the text line recognition result corresponding to each text line.

[0108] After correction, the text line content at each location is identified, and the text recognition result corresponding to each text line can be obtained.

[0109] Optionally, the text line correction image is input into the text recognition model, and the output result of the text recognition model is obtained to obtain the text recognition result corresponding to each text line.

[0110] The text recognition model is trained based on resume image samples and the corresponding text annotation information.

[0111] In some embodiments, the text recognition model can employ a CRNN (Convolutional Recurrent Neural Network) model, capable of recognizing longer text sequences. The CRNN model includes a CNN feature extraction layer and a BLSTM sequence feature extraction layer, enabling end-to-end joint training. Utilizing BLSTM and CTC components to learn the contextual relationships in character images effectively improves text recognition accuracy, making the model more robust. During prediction, the front end uses a standard CNN network to extract features from the text image, then uses BLSTM to fuse the feature vectors to extract contextual features of the character sequence, obtaining the probability distribution of each feature column, and finally predicting the text sequence through a transcription layer (CTC rule).

[0112] It should be noted that the embodiments of the present invention may also employ other models capable of text recognition to perform text recognition on text line correction images, and the present invention does not impose specific limitations on this.

[0113] In this embodiment of the invention, after detecting text lines in the target resume image, the detected text lines are corrected to obtain a text line corrected image. Then, text recognition is performed on the text line corrected image, which can improve the accuracy of text recognition and ensure the integrity of resume information extraction, thereby avoiding the loss of information that may occur when extracting text lines in complex scenarios.

[0114] Figure 7 This is a flowchart illustrating the process of correcting each line of text in a target resume image according to an embodiment of the present invention, such as... Figure 7 As shown, in some embodiments of the present invention, step 212 includes:

[0115] Step 2121: Based on the position information of each text line, use the bounding rectangle method to obtain the line image corresponding to each text line in the target resume image;

[0116] Based on the position information of the outline points of each text line, an outer rectangle is drawn to obtain the corresponding line image of each text line.

[0117] Step 2122: Based on the line image corresponding to each text line, obtain the rotated image corresponding to each text line using the minimum bounding rectangle method;

[0118] It can be understood that by determining the minimum bounding rectangle of the line image corresponding to each line of text, the rotated image corresponding to the line of text can be obtained.

[0119] Step 2123: Calculate the rotation angle of each text line based on the rotation image corresponding to each text line;

[0120] Once the rotated image corresponding to each line of text is obtained, the angle between the rotated image and the horizontal direction can be calculated, which is the rotation angle of each line of text.

[0121] Step 2124: Based on the rotation angle of each text line, perform an affine transformation on each text line to obtain the line image corresponding to each text line after rotation correction;

[0122] Based on the rotation angle, the affine transformation matrix can be determined. Then, the affine transformation matrix of each module performs an affine transformation on each text line to obtain the line image corresponding to each text line after rotation correction.

[0123] Step 2125: Use a mask image to remove interference information from the line images corresponding to each line of text after rotation correction, to obtain the text line correction image.

[0124] Finally, interference information in the line images corresponding to each line of text after rotation correction is removed. Specifically, a mask image can be used to remove interference information, and finally the text line correction image, i.e. the clean image after correction, and the position information of each text line after correction, such as coordinate information, angle information, etc., are obtained.

[0125] In this embodiment of the invention, by determining the rotated image corresponding to each line of text, calculating the rotation angle, and performing an affine transformation based on the rotation angle, the line image corresponding to each line of text is obtained after rotation correction. Then, a mask image is used to remove interference information in the line image corresponding to each line of text after rotation correction. This can overcome problems such as incorrect shooting angle and complex background information, and is suitable for extracting resume information in complex scenarios.

[0126] In some embodiments of the present invention, step 220 includes:

[0127] Based on the position information of each text line, the height directional overlap between semantic blocks in the target resume image is determined;

[0128] Based on the height directional overlap between semantic blocks in the target resume image, the semantic blocks are sorted.

[0129] The text lines within each semantic block are sorted to obtain the sorted text line content.

[0130] To improve the readability of text recognition results and overcome ambiguity caused by reading order, this embodiment of the invention proposes, after obtaining the position information and text recognition results of each text line, to sort the text lines in the target resume image based on the position information and text recognition results of each text line, thereby obtaining the sorted text line content.

[0131] Since semantic blocks in a resume generally have clear boundaries, the sorting of text lines can be achieved by outputting semantic blocks.

[0132] The output of semantic blocks is achieved through the following steps:

[0133] First, based on the positional information of each text line, the height directional overlap between semantic blocks in the target resume image is determined.

[0134] Then, the semantic blocks are sorted based on the height directional overlap between semantic blocks in the target resume image;

[0135] Optionally, if the overlap in the height direction is less than a preset value, for example, less than 0.4, then sorting is performed according to the height direction; otherwise, sorting is performed according to the left and right directions.

[0136] The text lines within each semantic block are sorted to obtain the sorted text line content.

[0137] The output of semantic blocks can be achieved through a semantic block generation model, which is trained based on the position information samples of each text line, the text recognition result samples, and the corresponding semantic block samples. The semantic block generation model executes the above-mentioned semantic block output steps.

[0138] In this embodiment of the invention, by sorting the semantic blocks and then sorting the text lines in the semantic blocks, the sorting of each text line in the target resume image can be achieved, which can effectively improve the readability of the text recognition results and thus better extract resume information.

[0139] Figure 8 This is a schematic diagram illustrating the encoding and decoding process for sorted text lines provided in an embodiment of the present invention. Figure 8 As shown, in some embodiments of the present invention, step 230 includes:

[0140] Step 231: Based on the sorted text line content, obtain text features; based on the position information of each text line and the target resume image, obtain layout features corresponding to each text line; based on the target resume image, obtain image features.

[0141] Optionally, in this embodiment of the invention, the LayoutXLM model is used to encode the sorted text lines. The LayoutXLM model requires information from three different modalities: text, layout, and image, as input.

[0142] Therefore, it is necessary to obtain text features, layout features, and image features.

[0143] The sorted text lines are input into a text feature extraction model for feature extraction to obtain text features.

[0144] In one embodiment, the bert_wwm model can be used as a text feature extraction model, that is, the sorted text lines are input into the bert_wwm model to obtain text features.

[0145] Based on the position information of each text line and the target resume image, the layout features corresponding to each text line are obtained. The layout features are extracted from the original target resume image based on the position information of each text line.

[0146] Based on the target resume image, image features are obtained, which are determined according to the target resume image.

[0147] Step 232: Perform feature fusion on the text features, layout features, and image features to obtain the fused text line features;

[0148] Optionally, the text features, layout features, and image features are input into an encoding model for feature fusion to obtain fused text line features;

[0149] The encoding model is trained based on text feature samples, layout feature samples, image feature samples, and text line feature samples.

[0150] The encoding model uses an attention mechanism to fuse inputs from multiple modalities.

[0151] The text features, layout features, and image features are input into the encoding model for feature fusion to obtain the fused text line features, wherein the fused text line features are features based on text lines.

[0152] Step 233: Perform hierarchical relationship reasoning on the fused text line features to obtain the text line structured relationship of the target resume image;

[0153] Optionally, the fused text line features are input into a decoding model for hierarchical relationship reasoning to obtain the text line structured relationship of the target resume image.

[0154] It is understood that in this embodiment of the invention, the decoding model is used to perform hierarchical relationship reasoning on the fused text line features in order to obtain the text line structured relationship of the target resume image.

[0155] The decoding model is trained based on text line feature samples and the text line structure relationships corresponding to the text line feature samples.

[0156] Optionally, embodiments of the present invention employ a GRU (Gate Recurrent Unit) model structure to implement hierarchical relationship reasoning.

[0157] It should be noted that other models can also be used for decoding, and this invention does not impose specific limitations on them.

[0158] In this embodiment of the invention, text features, layout features, and image features are acquired and input into an encoding model for feature fusion to obtain fused text line features. The fused text line features are then input into a decoding model for hierarchical relationship reasoning, thereby obtaining the text line structured relationship of the target resume image. Subsequently, based on the text line structured information, different resume extraction rules or requirements can be set according to the recruitment needs of different recruiting companies, thereby extracting different resume information and overcoming various restrictions on resumes.

[0159] In some embodiments of the present invention, step 233 specifically includes:

[0160] Based on the fused text line features, the title, parent node, and relationship with the parent node corresponding to each text line are determined.

[0161] For each text line, determine the hierarchical information of each text line based on the corresponding title and the relationship between each text line and its parent node;

[0162] Based on the hierarchical information of each text line, the structured relationship of the text lines in the target resume image is obtained.

[0163] Specifically, based on the features of the fused text lines, the title corresponding to each text line, the parent node of each text line, and the relationship between each text line and its parent node are determined, including inclusion, parallel, or connection relationships.

[0164] Then, for each line of text, execute the following sequentially: Figure 9 The steps shown are as follows:

[0165] Step 900: Determine whether the title is a first-level title;

[0166] Step 901: If it is a level 1 heading, output the level information of the current text line as 1;

[0167] Step 902: If the title is not a first-level title, determine whether the title is a start title;

[0168] Step 903: If the title is "beginning", then output the level information of the current text line as 1;

[0169] Step 904: If the title is not the start, determine whether the relationship is a connection or a parallel relationship;

[0170] Step 905: If the relationship is a connection or a parallel relationship, then output the hierarchy information of the current text line as the hierarchy information of the parent node;

[0171] Step 906: If the relationship is not a connection or parallel relationship, then determine whether the relationship is an inclusion relationship;

[0172] Step 907: If the relationship is an inclusion relationship, then output the hierarchy information of the current text line as the hierarchy information of the parent node plus 1;

[0173] Step 908: If the relationship is not an inclusion relationship, determine whether the title is the end;

[0174] Step 909: If the title is the end, then output the level information of the current text line as -1;

[0175] After performing steps 901 to 909 above on each line of text, the hierarchical information of each line of text can be obtained.

[0176] Finally, based on the hierarchical information of each text line, the structured relationship of the text lines in the target resume image can be obtained.

[0177] The trained decoding model is able to perform the hierarchical relationship reasoning steps described above.

[0178] In this embodiment of the invention, the hierarchical relationship between text lines is inferred through the decoding process, and the text line structure information of the target resume image is obtained. Since the text line structure information is closely related to the structure of the resume, the method provided by this invention can obtain the structure information of resumes with any structure, thereby breaking through various restrictions on resumes, such as complex sentences, multi-layered nested content, English resumes, etc., and realizing the extraction of useful information from resumes with different characteristics and of various types, which can effectively improve the resume screening efficiency and reduce the screening error rate.

[0179] Based on the above embodiments, the method further includes:

[0180] Based on the text line structure relationship of the target resume image, the semantic block to which each text line belongs is verified to obtain the verification result;

[0181] The semantic block is processed based on the verification result.

[0182] This embodiment mainly considers the following scenarios:

[0183] In the first scenario, if the current line and the previous line are connected or parallel, it means that the current line and the previous line need to be merged into a semantic block.

[0184] Scenario two involves page breaks. Specifically, we can determine if page breaks exist by combining the original position information of the text lines. If page breaks exist, the text lines that cross pages need to be merged into a single semantic block.

[0185] Scenario 3 requires the use of page number information. In this case, the semantic block needs to be split.

[0186] Considering the above scenarios, after obtaining the text line structure relationship of the target resume image, the semantic blocks to which each text line belongs are verified based on the text line structure relationship of the target resume image to obtain the verification result; based on the verification result, the semantic blocks are processed. The processing includes merging into a single semantic block or splitting a semantic block.

[0187] The verification process includes: determining whether the current line and the previous line are connected or parallel; if so, merging the current line and the previous text line into a semantic block.

[0188] Based on the original position information of the text lines, determine whether there are any page breaks. If page breaks exist, merge the text lines that cross pages into a single semantic block.

[0189] Determine whether page number information is needed. If so, split the semantic block. Note that the semantic block should be split into two, but the hierarchical information should remain consistent.

[0190] The resume information extraction method provided by this invention verifies the semantic blocks to which each text line belongs based on the structured relationship of the text lines in the target resume image, and obtains the verification result; based on the verification result, the semantic blocks are processed to further effectively extract resume information, which can solve the limitations of existing solutions on resume format and can obtain resume information in complex scenarios.

[0191] The resume information extraction device provided by the present invention is described below. The resume information extraction device described below and the resume information extraction method described above can be referred to in correspondence.

[0192] Figure 10 A schematic diagram of the resume information extraction device provided by the present invention is shown below. Figure 10 As shown, the device includes:

[0193] The text detection and recognition module 1010 is used to perform text line detection and text recognition on the target resume image to obtain the position information of each text line in the target resume image and the text recognition result corresponding to each text line.

[0194] The text sorting module 1020 is used to sort the text lines in the target resume image based on the position information and text recognition results of each text line, so as to obtain the sorted text line content.

[0195] The encoding / decoding processing module 1030 is used to encode and decode the sorted text line content to obtain the text line structured information of the target resume image.

[0196] In some embodiments of the present invention, the text detection and recognition module 1010 includes:

[0197] The text detection submodule is used to perform text line detection on the target resume image to obtain the position information of each text line in the target resume image.

[0198] The text line correction submodule is used to correct each text line in the target resume image according to the position information of each text line, so as to obtain a text line correction image;

[0199] The text recognition submodule is used to perform text recognition on the text line correction image to obtain the text line recognition result corresponding to each text line.

[0200] In some embodiments of the present invention, the text line correction submodule is used for:

[0201] Based on the position information of each text line, the line image corresponding to each text line in the target resume image is obtained by using the bounding rectangle method.

[0202] Based on the line image corresponding to each text line, the rotated image corresponding to each text line is obtained using the minimum bounding rectangle method.

[0203] Based on the rotated image corresponding to each text line, calculate the rotation angle of each text line;

[0204] Based on the rotation angle of each text line, an affine transformation is performed on each text line to obtain the line image corresponding to each text line after rotation correction;

[0205] The interference information in the line image corresponding to each line of text after rotation correction is removed by using a mask image to obtain the text line correction image.

[0206] In some embodiments of the present invention, the text sorting module 1020 is used for:

[0207] Based on the position information of each text line, the height directional overlap between semantic blocks in the target resume image is determined;

[0208] Based on the height directional overlap between semantic blocks in the target resume image, the semantic blocks are sorted.

[0209] The text lines within each semantic block are sorted to obtain the sorted text line content.

[0210] In some embodiments of the present invention, the encoding / decoding processing module 1030 includes:

[0211] The feature extraction submodule is used to obtain text features based on the sorted text line content, obtain layout features corresponding to each text line based on the position information of each text line and the target resume image, and obtain image features based on the target resume image;

[0212] The encoding submodule is used to perform feature fusion on the text features, layout features and image features to obtain the fused text line features;

[0213] The decoding submodule is used to perform hierarchical relationship reasoning on the fused text line features to obtain the text line structured relationship of the target resume image.

[0214] In some embodiments of the present invention, the decoding submodule is used for:

[0215] Based on the fused text line features, the title, parent node, and relationship with the parent node corresponding to each text line are determined.

[0216] For each text line, determine the hierarchical information of each text line based on the corresponding title and the relationship between each text line and its parent node;

[0217] Based on the hierarchical information of each text line, the structured relationship of the text lines in the target resume image is obtained.

[0218] In some embodiments of the present invention, the resume information extraction device further includes a verification module, the verification module being used for:

[0219] Based on the text line structure relationship of the target resume image, the semantic block to which each text line belongs is verified to obtain the verification result;

[0220] The semantic block is processed based on the verification result.

[0221] It should be noted that the resume information extraction device provided in this embodiment of the invention can implement all the method steps implemented in the above-mentioned resume information extraction method embodiment and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0222] Figure 11 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 11 As shown, the electronic device may include a processor 1110, a communications interface 1120, a memory 1130, and a communication bus 1140, wherein the processor 1110, the communications interface 1120, and the memory 1130 communicate with each other via the communication bus 1140. The processor 1110 can call logical instructions in the memory 1130 to execute a resume information extraction method. This method includes: performing text line detection and text recognition on a target resume image to obtain the position information of each text line in the target resume image and the text recognition result corresponding to each text line; sorting each text line in the target resume image based on the position information and text recognition result to obtain sorted text line content; and encoding / decoding the sorted text line content to obtain the text line structured information of the target resume image.

[0223] Furthermore, the logical instructions in the aforementioned memory 1130 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0224] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the resume information extraction method provided by the above methods. The method includes: performing text line detection and text recognition on a target resume image to obtain the position information of each text line in the target resume image and the text recognition result corresponding to each text line; sorting each text line in the target resume image based on the position information and text recognition result to obtain the sorted text line content; and encoding and decoding the sorted text line content to obtain the text line structured information of the target resume image.

[0225] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the resume information extraction method provided by the above methods. The method includes: performing text line detection and text recognition on a target resume image to obtain the position information of each text line in the target resume image and the text recognition result corresponding to each text line; sorting each text line in the target resume image based on the position information and text recognition result to obtain the sorted text line content; and encoding and decoding the sorted text line content to obtain the text line structured information of the target resume image.

[0226] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0227] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0228] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for extracting resume information, characterized in that, include: The target resume image is subjected to text line detection and text recognition to obtain the position information of each text line in the target resume image and the text recognition result corresponding to each text line. Based on the position information of each text line, the height directional overlap between semantic blocks in the target resume image is determined; Based on the height directional overlap between semantic blocks in the target resume image, the semantic blocks are sorted. The text lines within each semantic block are sorted to obtain the sorted text line content. The sorted text lines are encoded and decoded to obtain the text line structured information of the target resume image; The step of sorting the semantic blocks based on the height-direction overlap between semantic blocks in the target resume image includes: If the overlap in the height direction is less than a preset value, the semantic blocks are sorted according to the height direction; otherwise, the semantic blocks are sorted according to the left-right direction.

2. The resume information extraction method according to claim 1, characterized in that, The step of performing text line detection and text recognition on the target resume image to obtain the position information of each text line in the target resume image and the text recognition result corresponding to each text line includes: Perform text line detection on the target resume image to obtain the position information of each text line in the target resume image; Based on the position information of each text line, each text line in the target resume image is corrected to obtain a text line corrected image; Text recognition is performed on the text line correction image to obtain the text line recognition result corresponding to each text line.

3. The resume information extraction method according to claim 2, characterized in that, The step of correcting each text line in the target resume image based on the position information of each text line to obtain a text line corrected image includes: Based on the position information of each text line, the line image corresponding to each text line in the target resume image is obtained by using the bounding rectangle method. Based on the line image corresponding to each text line, the rotated image corresponding to each text line is obtained using the minimum bounding rectangle method. Based on the rotated image corresponding to each text line, calculate the rotation angle of each text line; Based on the rotation angle of each text line, an affine transformation is performed on each text line to obtain the line image corresponding to each text line after rotation correction; The interference information in the line image corresponding to each line of text after rotation correction is removed by using a mask image to obtain the text line correction image.

4. The resume information extraction method according to claim 1, characterized in that, The process of encoding and decoding the sorted text lines to obtain the text line structured information of the target resume image includes: Based on the sorted text line content, text features are obtained; based on the position information of each text line and the target resume image, layout features corresponding to each text line are obtained; and based on the target resume image, image features are obtained. The text features, layout features, and image features are fused to obtain the fused text line features; Hierarchical relationship reasoning is performed on the fused text line features to obtain the structured relationship of the text lines in the target resume image.

5. The resume information extraction method according to claim 4, characterized in that, The step of performing hierarchical relationship reasoning on the fused text line features to obtain the structured text line relationships of the target resume image includes: Based on the fused text line features, the title, parent node, and relationship with the parent node corresponding to each text line are determined. For each text line, determine the hierarchical information of each text line based on the corresponding title and the relationship between each text line and its parent node; Based on the hierarchical information of each text line, the structured relationship of the text lines in the target resume image is obtained.

6. The resume information extraction method according to claim 1, characterized in that, The method further includes: Based on the text line structure relationship of the target resume image, the semantic block to which each text line belongs is verified to obtain the verification result; The semantic block is processed based on the verification result.

7. A resume information extraction device, characterized in that, include: The text detection and recognition module is used to perform text line detection and text recognition on the target resume image to obtain the position information of each text line in the target resume image and the text recognition result corresponding to each text line. The text sorting module is used to determine the height direction overlap between semantic blocks in the target resume image based on the position information of each text line; sort the semantic blocks based on the height direction overlap between the semantic blocks in the target resume image; and sort the text lines within each sorted semantic block to obtain the sorted text line content. The encoding and decoding processing module is used to encode and decode the sorted text line content to obtain the text line structured information of the target resume image; The step of sorting the semantic blocks based on the height-direction overlap between semantic blocks in the target resume image includes: If the overlap in the height direction is less than a preset value, the semantic blocks are sorted according to the height direction; otherwise, the semantic blocks are sorted according to the left-right direction.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the resume information extraction method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the resume information extraction method as described in any one of claims 1 to 6.

Images