Resume information extraction method, device, equipment and storage medium

Abstract

The present invention relates to the field of data processing technology, and discloses a resume information extraction method, device, equipment and storage medium, the method comprising: obtaining a resume image of a current resume; preprocessing the resume image to obtain resume features corresponding to the current resume, the resume features including text information and layout information; performing multimodal analysis on the text information and layout information through a multimodal neural network to extract resume key information of the current resume. Since the present invention adopts a multimodal neural network, for a visually rich document such as a resume, the text information and layout information therein can be accurately analyzed and multimodal analysis can be performed to extract the resume key information contained in the current resume, thereby avoiding the existing resume analysis model based on deep learning, which may cause a large amount of information loss when extracting features from the resume, thereby accurately extracting key information and storing it in a structured talent pool, greatly saving labor costs and time costs.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a resume information extraction method, device, equipment and storage medium. Background Art

[0002] For enterprises, building a talent pool is very important in the process of human resource management. However, enterprises receive a large number of resumes every year. Generally, it takes time and money to remove duplicates, screen, and update the resumes accumulated in the talent pool, which is a large workload. As a result, it is difficult to effectively organize the talent pool and form intuitive structured data.

[0003] In order to solve the above problems, existing intelligent resume analysis technology uses natural language processing technology to extract the text of resumes of various file types, and then extracts information from the resume text to form structured information including basic personal information, education experience, work experience, skills, etc. However, intelligent resume analysis solutions are generally based on deep learning solutions. When extracting features from resumes, it generally leads to a large loss of information and it is difficult to obtain accurate key information.

[0004] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of the present invention is to provide a resume information extraction method, device, equipment and storage medium, aiming to solve the technical problem that the existing intelligent resume analysis solutions are generally based on deep learning solutions, which generally lead to large information loss when extracting features from resumes and make it difficult to obtain accurate key information.

[0006] To achieve the above object, the present invention provides a resume information extraction method, which comprises the following steps:

[0007] Get the resume image of the current resume;

[0008] Preprocessing the resume image to obtain resume features corresponding to the current resume, wherein the resume features include text information and layout information;

[0009] The text information and the layout information are subjected to multimodal analysis by a multimodal neural network to extract key resume information of the current resume.

[0010] Optionally, the resume image is preprocessed to obtain resume features corresponding to the current resume, wherein the resume features include text information and layout information, including:

[0011] Preprocessing the resume image in units of text blocks to obtain a plurality of resume text blocks corresponding to the resume image;

[0012] Extracting text information from each of the resume text blocks;

[0013] The layout information of each of the resume text blocks in the resume image is obtained.

[0014] Optionally, the layout information includes pixel coordinate values, and obtaining the layout information of each resume text block in the resume image includes:

[0015] identifying four text corners of each of the resume text blocks;

[0016] The resume text blocks are positioned based on the four text corners to obtain pixel coordinate values ​​of the four text corners of each resume text block on the resume image.

[0017] Optionally, before performing multimodal analysis on the text information and the layout information by a multimodal neural network to extract key resume information of the current resume, the method further includes:

[0018] Based on the pixel coordinate value of the upper left corner of the resume text block, the resume text blocks are arranged to obtain a text block coordinate sequence;

[0019] Accordingly, the multimodal analysis of the text information and the layout information by a multimodal neural network to extract key resume information of the current resume includes:

[0020] The text information and the text block coordinate sequence are subjected to multimodal analysis by a multimodal neural network to extract key resume information of the current resume.

[0021] Optionally, the performing multimodal analysis on the text information and the text block coordinate sequence by a multimodal neural network to extract key resume information of the current resume includes:

[0022] Performing multimodal analysis on the text information and the text block coordinate sequence through a multimodal neural network to obtain text block features of the resume text block;

[0023] Performing feature analysis on the text block features through a decoding neural network to determine the resume element label corresponding to each resume text block;

[0024] Based on the resume element tags, key resume information of the current resume is extracted.

[0025] Optionally, performing multimodal analysis on the text information and the text block coordinate sequence by a multimodal neural network to obtain text block features of the resume text block includes:

[0026] Based on the text block coordinate sequence and the text information, aligning the resume text block with a corresponding image block in the resume image;

[0027] Serialize and encode the aligned resume text blocks and image blocks to obtain the resume multimodality;

[0028] The resume multimodality is analyzed by a multimodal neural network to obtain text block features of the resume text block.

[0029] Optionally, before obtaining the resume image of the current resume, the method further includes:

[0030] Perform feature analysis on a preset number of job application resumes in the business scenario to obtain overall resume features of the job application resumes;

[0031] The overall resume features are extracted according to the needs of the enterprise to obtain resume elements of the job application resume, and resume element tags are constructed based on the resume elements.

[0032] In addition, to achieve the above purpose, the present invention also proposes a resume information extraction device, the device comprising:

[0033] An image acquisition module is used to acquire a resume image of the current resume;

[0034] A preprocessing module, used to preprocess the resume image to obtain resume features corresponding to the current resume, wherein the resume features include text information and layout information;

[0035] The multimodal analysis module is used to perform multimodal analysis on the text information and the layout information through a multimodal neural network to extract key resume information of the current resume.

[0036] In addition, to achieve the above-mentioned purpose, the present invention also proposes a resume information extraction device, which includes: a memory, a processor, and a resume information extraction program stored in the memory and executable on the processor, wherein the resume information extraction program is configured to implement the steps of the resume information extraction method described above.

[0037] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, on which a resume information extraction program is stored. When the resume information extraction program is executed by a processor, the steps of the resume information extraction method described above are implemented.

[0038] The present invention first obtains a resume image of the current resume; then preprocesses the resume image to obtain resume features corresponding to the current resume, the resume features including text information and layout information; finally, a multimodal analysis is performed on the text information and the layout information through a multimodal neural network to extract resume key information of the current resume. Since the present invention uses a multimodal neural network, it can accurately analyze the text information and layout information of a visually rich document such as a resume and perform multimodal analysis to extract resume key information contained in the current resume, thereby avoiding the existing resume analysis model based on deep learning, which may cause a large amount of information loss when extracting features from the resume, thereby accurately extracting key information and storing it in a structured talent pool, greatly saving labor costs and time costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a structural schematic diagram of a resume information extraction device in a hardware operating environment involved in an embodiment of the present invention;

[0040] Figure 2 This is a flow chart of the first embodiment of the resume information extraction method of the present invention;

[0041] Figure 3 A schematic diagram of extracting resume features based on deep learning in the first embodiment of the resume information extraction method of the present invention;

[0042] Figure 4 This is a flow chart of a second embodiment of the resume information extraction method of the present invention;

[0043] Figure 5 It is a schematic diagram of extracting text block features by a multimodal neural network in the second embodiment of the resume information extraction method of the present invention;

[0044] Figure 6 This is a structural block diagram of the first embodiment of the resume information extraction device of the present invention.

[0045] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0046] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0047] Reference Figure 1 , Figure 1 The present invention is a schematic diagram of the structure of a resume information extraction device in a hardware operating environment according to an embodiment of the present invention.

[0048] like Figure 1As shown, the resume information extraction device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (Wireless-Fidelity, Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM), or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk storage. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0049] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the resume information extraction device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0050] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a resume information extraction program.

[0051] exist Figure 1 In the resume information extraction device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the resume information extraction device of the present invention can be set in the resume information extraction device, and the resume information extraction device calls the resume information extraction program stored in the memory 1005 through the processor 1001, and executes the resume information extraction method provided by the embodiment of the present invention.

[0052] The embodiment of the present invention provides a resume information extraction method, referring to Figure 2 , Figure 2 Schematic diagram of the process of the first embodiment of the resume information extraction method of the present invention.

[0053] It should be noted that the current talent pool solutions in enterprises are mainly focused on the application of structured talent pools, which aggregate resume resources and accumulate enterprise talent assets based on the enterprise organizational structure. An intelligent talent pool is established on the analyzed resume database, including a multi-level talent pool system, automatic reserve and automatic update of resumes, intelligent search for talents, talent tag system and other applications to facilitate enterprise talent management.

[0054] However, the above schemes have not yet analyzed the original resumes, and manual analysis still requires a lot of manpower and time. Intelligent resume analysis technology generally uses natural language processing technology to extract the text of resumes of various file types, and then extracts information from the resume text to form structured information containing basic personal information, education experience, work experience, skills, etc. Among them, intelligent resume analysis schemes can be divided into three categories: dictionary-based and rule-based schemes, machine learning-based schemes, and deep learning-based schemes. The dictionary-based scheme first establishes a dictionary library, and then uses the dictionary library to match the corresponding resume information elements to realize text resume information extraction. The rule-based method uses the more obvious key-value relationship in the resume to write extraction rules and extract various entities. The machine learning-based extraction scheme mainly uses hidden Markov models (HMM) and support vector machine models (SVM). HMM is used to segment the resume into continuous text blocks, and then SVM is used to extract personal detailed information.

[0055] Deep learning can learn deeper feature representations of data. For the text features and layout features of resumes, two different neural networks are generally used to extract the features of the two aspects respectively, and then these features are integrated to analyze the resume information. Figure 3 , Figure 3 FIG. 1 is a schematic diagram of extracting resume features based on deep learning in the first embodiment of the resume information extraction method of the present invention. Figure 3 As shown in the figure, for the text features in the resume (such as "Li Lei graduated from Peking University"), first train the word vector to vectorize the input text sequence, and then use the recurrent convolutional neural network, bidirectional long short-term memory network and probabilistic graph model or pre-trained language model to parse the text in the resume; and for the layout features in the resume, digital image and computer technology are used to construct. After obtaining the two types of features, the corresponding fusion algorithm is designed to align them and perform feature fusion to parse the information entity in the resume.

[0056] However, the dictionary and rule-based resume information extraction method requires manual construction of dictionaries or formulation of relevant rules according to the extraction goals and requirements, and then matching using regular expressions. The accuracy of the extraction results depends on the quality of the dictionary and extraction rules. It is necessary to manually establish and maintain rules, and the robustness, generalization ability, and migration ability are poor, which has great limitations.

[0057] Research on resume information extraction based on machine learning reduces the cost of manual rule maintenance, but requires the preparation of a large amount of data. The accuracy is to a certain extent related to the quality and quantity of data selection, and its ability is limited when faced with the generalization problem of complex data.

[0058] The existing resume information extraction method based on deep learning extracts the text features and layout features of the resume separately, and then uses a certain method to fuse the two features. However, this method ignores the direct connection between text and layout when extracting features, and only uses the characteristics of the two information themselves for analysis when fusing features, resulting in a large loss of information and difficulty in improving the accuracy of the model.

[0059] In order to solve the above problem, this embodiment proposes a resume information extraction method.

[0060] In this embodiment, the resume information extraction method includes the following steps:

[0061] Step S10: Obtain a resume image of the current resume.

[0062] It should be noted that the execution subject of the method of this embodiment can be a computing service device with image processing, feature extraction and multimodal analysis functions, such as a personal computer, a server, etc., or other electronic devices that can achieve the same or similar functions, such as the above-mentioned resume information extraction device, and this embodiment does not limit this. Here, the above-mentioned resume information extraction device (referred to as the extraction device) is used to specifically describe this embodiment and the following embodiments.

[0063] It is understandable that the resume image is a picture display of the current resume. If it is a paper resume, it can be obtained by image scanning, and this embodiment does not impose any restrictions on this.

[0064] Step S20: pre-processing the resume image to obtain resume features corresponding to the current resume, wherein the resume features include text information and layout information.

[0065] It should be noted that resume features are key text information in a resume that demonstrates personal strengths, characteristics, and abilities, as well as information such as resume style.

[0066] It can be understood that text information is the text content contained in the resume, such as personal information, educational background, work experience, skills and expertise, project experience, honors and awards, self-evaluation, personal interests and hobbies, etc.

[0067] It should be understood that the layout information is the overall typesetting and structural arrangement of the resume. A good layout can make the resume look neat and clear, and easier for recruiters to browse and understand. The extraction of layout information can be determined based on the coordinates of each text information in the resume, or determined by relative position, which is not limited in this embodiment.

[0068] In a specific implementation, the extraction device can pre-process the resume image to obtain resume features corresponding to the current resume, wherein the resume features include text information such as personal information, educational background, work experience, hobbies, etc., as well as layout information of the overall resume layout and structural arrangement.

[0069] Step S30: Perform multimodal analysis on the text information and the layout information through a multimodal neural network to extract key resume information of the current resume.

[0070] It should be noted that a multimodal neural network is a neural network model that can process different types of input data. For example, the above-mentioned text information and layout information usually contain rich information, and it may be difficult to fully explore the relationship and common features between these information by processing each type of data separately. Multimodal neural networks can process these different types of data at the same time, thereby achieving more comprehensive and comprehensive information extraction and analysis.

[0071] In a specific implementation, the extraction device can perform multimodal analysis on the text information and the layout information through a multimodal neural network to fully explore the relationship and common features between the text information and the layout information, thereby extracting key resume information in the resume.

[0072] Furthermore, before step S10, this embodiment also includes: performing feature analysis on a preset number of job application resumes in the business scenario to obtain overall resume features of the job application resumes; extracting elements of the overall resume features according to enterprise needs to obtain resume elements of the job application resumes, and constructing resume element tags based on the resume elements.

[0073] It should be noted that the preset number is a pre-set number, such as 700 resumes, 800 resumes, etc. The overall resume characteristics are the overall characteristics and features of the resume, including the collective performance of content, format, layout and style.

[0074] It is understandable that resume elements are the basic elements in the resume content, such as name, phone number, email address, job intention, etc. Important elements can be extracted from the basic elements of the resume as element tags for resume analysis to improve the efficiency of resume analysis.

[0075] Specifically, resume feature analysis is mainly divided into the following three parts:

[0076] (1) Overall characteristics: The overall style and style of the resume will be affected by the job seeker and the job search platform, and the structure is relatively variable.

[0077] (2) Layout features: Resume information is usually not presented in the form of large paragraphs of text. Different layouts are designed according to the category and importance of each type of information. For example, the basic information of job seekers is usually on one page, while educational experience and project manager are generally on different pages. Currently, resume information is relatively rich. In addition to text information, visual information such as document layout can also help companies quickly extract the information they need.

[0078] (3) Resume elements: Companies will make a preliminary judgment on job seekers based on certain information in the resume. For example, after analyzing 700 data sets, the present invention summarizes 13 important elements in resumes. Their names and brief introductions are as follows:

[0079] Name: Name of the resume owner.

[0080] Phone: Phone information provided by the resume owner.

[0081] Email: Email information provided by the resume owner.

[0082] Job intention: The target position provided by the resume owner, such as "web front-end development", "Java engineer", etc.

[0083] School Name: The name of all the institutions that awarded the resume owner’s academic qualifications on the page, including high school, college, bachelor’s degree, master’s degree, and doctorate.

[0084] Study time: The time span during which the resume owner received education at the institution that issued his / her degree.

[0085] Major: The name of the major that the resume owner obtained when he / she graduated. If he / she has not graduated yet, the name of the major he / she is currently studying.

[0086] Project Name: Names of all projects the resume owner has participated in.

[0087] Project time: The time the resume owner participated in the project.

[0088] Position in the project: The position that the resume owner is responsible for in the project he / she participates in.

[0089] Workplace: The name of the company where the resume owner has worked (interns) for all times.

[0090] Working time: The time the resume owner has been working.

[0091] Job position: The job position of the resume owner when he or she started working.

[0092] In a specific implementation, the extraction device can perform feature analysis on a preset number of job application resumes in a business scenario to obtain the above-mentioned overall resume features; then, perform element extraction on the overall resume features according to enterprise needs to obtain resume elements of the job application resume, such as name, telephone number, email address, job application intention, etc. Finally, important elements can be extracted from the basic elements of the resume as element tags for resume analysis to improve the efficiency of resume analysis.

[0093] The extraction device of this embodiment can first obtain the resume image of the current resume, and then pre-process the resume image to obtain the resume features corresponding to the current resume, and the resume features include text information such as personal information, educational background, work experience, hobbies and other text content, as well as the layout information of the overall layout and structural arrangement of the resume. Finally, the text information and the layout information can be multimodally analyzed by a multimodal neural network to fully explore the relationship and common features between the text information and the layout information, so as to extract the resume key information in the resume. Since this embodiment adopts a multimodal neural network, for a rich visual document such as a resume, the text information and layout information therein can be accurately analyzed and multimodally analyzed, and the resume key information contained in the current resume can be extracted, thereby avoiding the existing resume analysis model based on deep learning. When extracting features from a resume, it may cause a large amount of information loss, so that key information can be accurately extracted and stored in a structured talent pool, greatly saving labor costs and time costs.

[0094] refer to Figure 4 , Figure 4 Schematic diagram of the flow of the second embodiment of the resume information extraction method of the present invention.

[0095] Based on the above first embodiment, in this embodiment, step S20 includes:

[0096] Step S21: pre-processing the resume image in units of text blocks to obtain multiple resume text blocks corresponding to the resume image.

[0097] It should be noted that resume text blocks are text blocks that are divided into text blocks when processing resume images. By processing resume images into multiple resume text blocks, text information can be extracted from resume images for subsequent text recognition, information extraction or data analysis tasks, thereby improving the efficiency of resume processing.

[0098] Step S22: extracting text information from each resume text block.

[0099] Step S23: Obtaining layout information of each resume text block in the resume image.

[0100] It should be noted that the layout information is the overall typesetting and structural arrangement of the resume. The extraction of the layout information can be determined based on the coordinates of each text information in the resume, or determined by relative position, which is not limited in this embodiment.

[0101] In a specific implementation, the extraction device first pre-processes the resume image in units of text blocks to obtain multiple resume text blocks corresponding to the resume image; then extracts the text information and layout information of each resume text block for subsequent information processing. By processing the resume image into multiple resume text blocks, text information can be extracted from the resume image for subsequent text recognition, information extraction or data analysis tasks, thereby improving the efficiency of resume processing.

[0102] Furthermore, the layout information includes pixel coordinate values. In this embodiment, step S23 includes: identifying the four text corners of each resume text block; positioning the resume text block based on the four text corners, and obtaining the pixel coordinate values ​​of the four text corners of each resume text block on the resume image.

[0103] It should be noted that the pixel coordinate value is the unit used to represent the position of the resume text block in the resume image. In the resume image, each point can be determined by a pixel coordinate value. For example, the upper left corner of the resume image is taken as the origin (0,0), the horizontal direction to the right is the positive direction of the x-axis, and the vertical direction downward is the positive direction of the y-axis, to construct the coordinate axis of the resume image. Based on this coordinate axis, the pixel coordinate values ​​of the four text corners of each resume text block on the resume image can be obtained.

[0104] In a specific implementation, the extraction device can first construct the coordinate axis of the resume image with the upper left corner of the resume image as the origin (0,0), the horizontal direction to the right as the positive direction of the x-axis, and the vertical direction downward as the positive direction of the y-axis. Then identify the four text corners of each resume text block; coordinate the resume text block in the coordinate axis according to the four text corners, and obtain the pixel coordinate values ​​of the four text corners of each resume text block on the resume image. Thus, the layout information of the resume image can be quickly and accurately determined through the pixel coordinate values.

[0105] Furthermore, in this embodiment, before step S30, it also includes: arranging each resume text block based on the upper left corner pixel coordinate value of the resume text block to obtain a text block coordinate sequence; correspondingly, the step S30 includes: performing multimodal analysis on the text information and the text block coordinate sequence through a multimodal neural network to extract the resume key information of the current resume.

[0106] It should be noted that the text block coordinate sequence refers to the sequence formed by arranging the position information of the resume text block in a certain order when processing the resume text block. The text block coordinate sequence can be used to represent the layout and arrangement of the text in the image, restore the paragraph structure in the document, and facilitate the positioning and recognition of the text in the image according to the text block coordinate sequence.

[0107] In actual implementation, the preprocessing of the current resume document needs to be analyzed from two aspects: text information and layout information. First, the resume is preprocessed in units of text blocks, and the text content and pixel coordinate values ​​of the four corners of each text block in the resume image are obtained. Then, the text blocks are arranged according to the coordinate value of the upper left corner of the text block to form a text sequence and a corresponding coordinate sequence for subsequent multimodal analysis. Thus, the layout information of the resume image can be quickly and accurately determined through the pixel coordinate value.

[0108] Furthermore, the present embodiment performs multimodal analysis on the text information and the text block coordinate sequence through a multimodal neural network to extract resume key information of the current resume, including: performing multimodal analysis on the text information and the text block coordinate sequence through a multimodal neural network to obtain text block features of the resume text block; performing feature analysis on the text block features through a decoding neural network to determine the resume element labels corresponding to each resume text block; and extracting resume key information of the current resume based on the resume element labels.

[0109] It should be noted that the decoding neural network is the part of the neural network model used to convert the model output into the final result. The decoding neural network can perform feature analysis on the text block features and convert them into readable results, thereby determining the resume element labels corresponding to each resume text block.

[0110] In a specific implementation, the extraction device can perform multimodal analysis on the text information and the text block coordinate sequence through a multimodal neural network to obtain the text block features of the resume text block. Then, the text block features are analyzed through a decoding neural network, and a corresponding label is assigned to each text block, thereby extracting the key information in the resume.

[0111] Furthermore, the present embodiment performs multimodal analysis on the text information and the text block coordinate sequence through a multimodal neural network to obtain text block features of the resume text block, including: aligning the resume text block with the corresponding image block in the resume image based on the text block coordinate sequence and the text information; serializing and encoding the aligned resume text block and image block to obtain resume multimodality; and analyzing the resume multimodality through a multimodal neural network to obtain text block features of the resume text block.

[0112] In actual implementation, refer to Figure 5 , Figure 5 FIG. 1 is a schematic diagram of a multimodal neural network extracting text block features in the second embodiment of the resume information extraction method of the present invention. Figure 5 As shown, the resume document needs to be analyzed from two aspects: text information and layout information. It is divided into text blocks (such as name, nationality, etc.) and image blocks about layout information. After the text blocks are arranged according to the coordinate values ​​of the upper left corner of the text blocks to form a text sequence and a corresponding coordinate sequence, multimodal processing of the resume begins. According to the coordinates of the text block, its corresponding image block can be found in the image, and then the text block is aligned with the image block, and serialized encoding is performed at the same time (as shown in the figure, T1, T2, T3, T4 or V1, V2, V3, V4). Then the resume text block is aligned with the corresponding image block in the resume image. Finally, a multimodal neural network is used to analyze the two modes and extract the characteristic text block features of each text block.

[0113] The extraction device of this embodiment first pre-processes the resume image in units of text blocks to obtain multiple resume text blocks corresponding to the resume image; then extracts the text information and layout information of each resume text block for subsequent information processing. By processing the resume image into multiple resume text blocks, text information can be extracted from the resume image to facilitate subsequent text recognition, information extraction or data analysis tasks, thereby improving the efficiency of resume processing. Furthermore, the extraction device can also first construct the coordinate axis of the resume image with the upper left corner of the resume image as the origin (0,0), the horizontal direction to the right as the positive direction of the x-axis, and the vertical direction downward as the positive direction of the y-axis. Then identify the four text corners of each resume text block; coordinate the resume text block in the coordinate axis according to the four text corners, and obtain the pixel coordinate values ​​of the four text corners of each resume text block on the resume image. Thus, the layout information of the resume image can be quickly and accurately determined through the pixel coordinate values. Furthermore, the current resume document preprocessing needs to be analyzed from two aspects: text information and layout information. First, the resume is preprocessed in units of text blocks, and the text content and pixel coordinate values ​​of the four corners of each text block in the resume image are obtained. Then, the text blocks are arranged according to the coordinate value of the upper left corner of the text block to form a text sequence and a corresponding coordinate sequence for subsequent multimodal analysis. Thus, the layout information of the resume image can be quickly and accurately determined through the pixel coordinate value.

[0114] In addition, an embodiment of the present invention further proposes a storage medium, on which a resume information extraction program is stored. When the resume information extraction program is executed by a processor, the steps of the resume information extraction method described above are implemented.

[0115] Reference Figure 6 , Figure 6This is a structural block diagram of the first embodiment of the resume information extraction device of the present invention.

[0116] like Figure 6 As shown, the resume information extraction device proposed in the embodiment of the present invention includes:

[0117] An image acquisition module 601 is used to acquire a resume image of the current resume;

[0118] A preprocessing module 602 is used to preprocess the resume image to obtain resume features corresponding to the current resume, where the resume features include text information and layout information;

[0119] The multimodal analysis module 603 is used to perform multimodal analysis on the text information and the layout information through a multimodal neural network to extract key resume information of the current resume.

[0120] The extraction device of this embodiment can first obtain the resume image of the current resume, and then pre-process the resume image to obtain the resume features corresponding to the current resume, and the resume features include text information such as personal information, educational background, work experience, hobbies and other text content, as well as the layout information of the overall layout and structural arrangement of the resume. Finally, the text information and the layout information can be multimodally analyzed by a multimodal neural network to fully explore the relationship and common features between the text information and the layout information, so as to extract the resume key information in the resume. Since this embodiment adopts a multimodal neural network, for a rich visual document such as a resume, the text information and layout information therein can be accurately analyzed and multimodally analyzed, and the resume key information contained in the current resume can be extracted, thereby avoiding the existing resume analysis model based on deep learning. When extracting features from a resume, it may cause a large amount of information loss, so that key information can be accurately extracted and stored in a structured talent pool, greatly saving labor costs and time costs.

[0121] Based on the above-mentioned first embodiment of the resume information extraction device of the present invention, a second embodiment of the resume information extraction device of the present invention is proposed.

[0122] In this embodiment, the preprocessing module 602 is also used to preprocess the resume image in units of text blocks to obtain multiple resume text blocks corresponding to the resume image; extract text information of each resume text block; and obtain layout information of each resume text block in the resume image.

[0123] Furthermore, the layout information includes pixel coordinate values, and the preprocessing module 602 is also used to identify the four text corners of each resume text block; coordinate the resume text block based on the four text corners, and obtain the pixel coordinate values ​​of the four text corners of each resume text block on the resume image.

[0124] Furthermore, the resume information extraction device further includes a coordinate arrangement module 604, which is used to arrange each resume text block based on the upper left corner pixel coordinate value of the resume text block to obtain a text block coordinate sequence;

[0125] Correspondingly, the multimodal analysis module 603 is also used to perform multimodal analysis on the text information and the text block coordinate sequence through a multimodal neural network to extract the resume key information of the current resume.

[0126] Furthermore, the multimodal analysis module 603 is also used to perform multimodal analysis on the text information and the text block coordinate sequence through a multimodal neural network to obtain text block features of the resume text block; perform feature analysis on the text block features through a decoding neural network to determine the resume element labels corresponding to each resume text block; and extract resume key information of the current resume based on the resume element labels.

[0127] Furthermore, the multimodal analysis module 603 is also used to align the resume text block with the corresponding image block in the resume image based on the text block coordinate sequence and the text information; serialize and encode the aligned resume text block and image block to obtain resume multimodality; and analyze the resume multimodality through a multimodal neural network to obtain text block features of the resume text block.

[0128] Furthermore, the resume information extraction device also includes an element label module 605, which is used to perform feature analysis on a preset number of job application resumes in a business scenario to obtain overall resume features of the job application resumes; extract elements of the overall resume features according to enterprise needs to obtain resume elements of the job application resume, and construct resume element labels based on the resume elements.

[0129] Other embodiments or specific implementations of the resume information extraction device of the present invention can refer to the above-mentioned method embodiments and will not be described in detail here.

[0130] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0131] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0132] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0133] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A resume information extraction method, characterized in that: The resume information extraction method comprises: Get the resume image of the current resume; Preprocessing the resume image to obtain resume features corresponding to the current resume, wherein the resume features include text information and layout information; Performing multimodal analysis on the text information and the layout information through a multimodal neural network to extract key resume information of the current resume; The preprocessing of the resume image to obtain resume features corresponding to the current resume, wherein the resume features include text information and layout information, includes: preprocessing the resume image in units of text blocks to obtain multiple resume text blocks corresponding to the resume image; extracting text information of each resume text block; obtaining layout information of each resume text block in the resume image; Among them, before the multimodal analysis of the text information and the layout information is performed through a multimodal neural network to extract the resume key information of the current resume, it also includes: arranging each resume text block based on the upper left corner pixel coordinate value of the resume text block to obtain a text block coordinate sequence; correspondingly, the multimodal analysis of the text information and the layout information is performed through a multimodal neural network to extract the resume key information of the current resume, including: performing a multimodal analysis of the text information and the text block coordinate sequence through a multimodal neural network to extract the resume key information of the current resume; The method of performing multimodal analysis on the text information and the text block coordinate sequence by a multimodal neural network to extract the resume key information of the current resume includes: performing multimodal analysis on the text information and the text block coordinate sequence by a multimodal neural network to obtain the text block features of the resume text block; performing feature analysis on the text block features by a decoding neural network to determine the resume element labels corresponding to each resume text block; and extracting the resume key information of the current resume based on the resume element labels. Among them, the multimodal analysis of the text information and the text block coordinate sequence by a multimodal neural network to obtain the text block features of the resume text block includes: aligning the resume text block with the corresponding image block in the resume image based on the text block coordinate sequence and the text information; serializing and encoding the aligned resume text block and image block to obtain resume multimodality; analyzing the resume multimodality by a multimodal neural network to obtain the text block features of the resume text block.

2. The resume information extraction method according to claim 1, characterized in that: The layout information includes pixel coordinate values, and the step of obtaining the layout information of each resume text block in the resume image includes: identifying four text corners of each of the resume text blocks; The resume text blocks are positioned based on the four text corners to obtain pixel coordinate values ​​of the four text corners of each resume text block on the resume image.

3. The resume information extraction method according to any one of claims 1 to 2, characterized in that: Before obtaining the resume image of the current resume, the method further includes: Perform feature analysis on a preset number of job application resumes in the business scenario to obtain overall resume features of the job application resumes; The overall resume features are extracted according to the needs of the enterprise to obtain resume elements of the job application resume, and resume element tags are constructed based on the resume elements.

4. A resume information extraction device, characterized in that: The resume information extraction device applies the resume information extraction method according to claim 1, and the device comprises: An image acquisition module is used to acquire a resume image of the current resume; A preprocessing module, used to preprocess the resume image to obtain resume features corresponding to the current resume, wherein the resume features include text information and layout information; The multimodal analysis module is used to perform multimodal analysis on the text information and the layout information through a multimodal neural network to extract key resume information of the current resume.

5. A resume information extraction device, characterized in that: The device comprises: a memory, a processor and a resume information extraction program stored in the memory and executable on the processor, wherein the resume information extraction program is configured to implement the steps of the resume information extraction method according to any one of claims 1 to 3.

6. A storage medium, characterized in that: The storage medium stores a resume information extraction program, and when the resume information extraction program is executed by the processor, the steps of the resume information extraction method according to any one of claims 1 to 3 are implemented.

Images