Resume disambiguation method, device, equipment, medium and product
By constructing and decomposing the resume matrix, calculating and clustering resume users and university networks, the problem of low accuracy of resume disambiguation in the existing technology is solved, and a more efficient disambiguation effect of the same name is achieved.
Patent Information
- Application Number
- CN202411985965.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-31
- Publication Date
- 2025-04-29
AI Technical Summary
The prior art has low accuracy when dealing with resume disambiguation of entities with the same name and cannot effectively deal with complex situations.
By collecting the teacher’s homepage address, building a resume matrix, and performing singular value decomposition, the feature matrix, content feature matrix and diagonal matrix of resume users are obtained. These submatrix calculations are used to obtain resume user networks and university networks, cluster, integrate outlier resumes, and obtain a collection of disambiguous resumes.
It improves the accuracy of resume disambiguation, captures complex relationship information between resumes, and improves the effect of disambiguation of the same name.
Smart Images
Figure CN119903833A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of natural language processing technology, and in particular to a resume disambiguation method, device, equipment, medium and product. Background Art
[0002] Author Name Disambiguation (AND) is an important but challenging task in scientific research and academia. With the increase in the number of university faculty and the continuous expansion of resume databases, it has become increasingly difficult to accurately identify and distinguish different faculty members with the same or similar names. Solving this problem is of great significance to the accuracy of university resume databases and the construction of scientific research cooperation networks.
[0003] At present, research on name disambiguation has made some progress. The methods of homonym disambiguation mainly include rule-based matching and clustering algorithms. However, these methods often rely on static features of resume information (such as name, gender, date of birth, etc.) or a single representation learning strategy, which cannot effectively handle the complex situations encountered by homonymous entities in practical applications, resulting in low accuracy of resume disambiguation. Summary of the invention
[0004] The purpose of this application is to provide a resume disambiguation method, device, equipment, medium and product, which can improve the accuracy of resume disambiguation.
[0005] To achieve the above objectives, this application provides the following solutions:
[0006] In a first aspect, the present application provides a resume disambiguation method, comprising:
[0007] Determine a resume information set according to the collected teacher homepage address; wherein the resume information set includes resume information corresponding to multiple teachers respectively; the resume information includes at least a resume ID, a teacher's name, work information, and education information;
[0008] Constructing a resume matrix according to the resume information set; wherein the resume matrix includes a plurality of matrix rows and a plurality of matrix columns, one matrix row corresponds to one resume information, and one matrix column corresponds to one pre-stored university information; the matrix value in the resume matrix represents the correlation between the resume information corresponding to the matrix value and the university information corresponding to the matrix value;
[0009] Performing singular value decomposition on the resume matrix to obtain multiple sub-matrices; wherein the multiple sub-matrices at least include a resume user's feature matrix, a content feature matrix, and a diagonal matrix;
[0010] Using the multiple sub-matrices, a resume user network and a resume university network are calculated;
[0011] Clustering the resume user network and the resume university network to obtain a pre-clustered resume set and an outlier resume set;
[0012] Based on the pre-clustered resume set, the outlier resume set is integrated to obtain a disambiguated resume set.
[0013] Optionally, determining a resume information set based on the collected teacher homepage address specifically includes:
[0014] Obtaining the homepage text information of multiple teachers from the collected teacher homepage addresses;
[0015] Extracting the preset key information in each of the homepage text information to obtain the initial resume information corresponding to each of the homepage text information;
[0016] Preprocess each of the initial resume information to obtain resume information corresponding to each of the initial resume information;
[0017] Integrate each resume information to obtain a resume information set.
[0018] Optionally, the feature matrix of the resume user is a representation of the resume user in a potential feature space; each row in the feature matrix of the resume user corresponds to a resume user of resume information;
[0019] The content feature matrix is a representation of the university information in the potential feature space.
[0020] Optionally, using the multiple sub-matrices to calculate the resume user network and the resume university network specifically includes:
[0021] Using the feature matrix of the resume user, a resume user network is calculated;
[0022] Determine a feature space representation of the resume user using the feature matrix of the resume user and the diagonal matrix;
[0023] Determine the feature space representation of the university information using the content feature matrix and the diagonal matrix;
[0024] A resume-university network is determined using the feature space representation of the resume user and the feature space representation of the university information.
[0025] Optionally, the using the feature matrix of the resume user to calculate the resume user network specifically includes:
[0026] Based on the feature matrix of the resume users, calculate the cosine similarity between the rows of every two resume users;
[0027] Establish a target edge between two target resume users whose inter-row cosine similarity is greater than a preset threshold;
[0028] Based on the resume users and the target edges, a resume user network is constructed.
[0029] Optionally, the pre-clustered resume set includes multiple pre-clustered resume clusters, each pre-clustered resume cluster includes multiple pre-clustered resumes, the outlier resume combination includes outlier resumes, and the outlier resume set is integrated based on the pre-clustered resume set to obtain the disambiguated resume set, specifically including:
[0030] Acquire a target outlier resume from the outlier resume set;
[0031] Calculate and obtain a first similarity between the target outlier resume and each pre-clustered resume;
[0032] If the maximum first similarity is greater than or equal to a preset threshold, the target outlier resume is moved from the outlier resume set to the pre-clustered resume cluster where the target pre-clustered resume corresponding to the maximum first similarity is located;
[0033] If the maximum first similarity is less than the preset threshold, constructing a new pre-clustered resume cluster in the pre-clustered resume set, and moving the target outlier resume from the outlier resume set to the new pre-clustered resume cluster;
[0034] Calculate and obtain a second similarity between the target outlier resume and each current outlier resume in the outlier resume set;
[0035] Determine a target current outlier resume from the outlier resume set; wherein the second similarity corresponding to the target current outlier resume is greater than or equal to the preset threshold;
[0036] Move the target current outlier resume from the outlier resume set to the pre-clustered resume cluster where the target outlier resume is located;
[0037] If there are still outlier resumes in the outlier resume set, executing the step from obtaining the target outlier resume from the outlier resume set to moving the target current outlier resume from the outlier resume set to the pre-clustered resume cluster where the target outlier resume is located;
[0038] If there is no outlier resume in the outlier resume set, the current pre-clustered resume set is determined as the disambiguation resume set.
[0039] In a second aspect, the present application provides a resume disambiguation device, comprising:
[0040] A determination unit, configured to determine a resume information set according to the collected teacher homepage address; wherein the resume information set includes resume information corresponding to a plurality of teachers respectively; and the resume information includes at least a resume ID, a teacher's name, work information, and education information;
[0041] A construction unit is used to construct a resume matrix according to the resume information set; wherein the resume matrix includes a plurality of matrix rows and a plurality of matrix columns, one matrix row corresponds to one resume information, and one matrix column corresponds to one pre-stored university information; the matrix value in the resume matrix represents the correlation between the resume information corresponding to the matrix value and the university information corresponding to the matrix value;
[0042] A decomposition unit, used to perform singular value decomposition on the resume matrix to obtain multiple sub-matrices; wherein the multiple sub-matrices at least include a feature matrix of the resume user, a content feature matrix, and a diagonal matrix;
[0043] A calculation unit, used to calculate the resume user network and the resume university network using the multiple sub-matrices;
[0044] A clustering unit, used for clustering the resume user network and the resume university network to obtain a pre-clustered resume set and an outlier resume set;
[0045] An integration unit is used to integrate the outlier resume set based on the pre-clustered resume set to obtain a disambiguated resume set.
[0046] In a third aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the resume disambiguation methods described above.
[0047] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the resume disambiguation methods described above.
[0048] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of any of the resume disambiguation methods described above.
[0049] According to the specific embodiments provided in this application, this application discloses the following technical effects:
[0050] The present application provides a resume disambiguation method, apparatus, equipment, medium and product, which can construct a resume matrix based on a collected resume information set, and obtain multiple sub-matrices by performing singular value decomposition on the resume matrix. This method can capture the complex relationship information between resumes and improve the accuracy of homonym disambiguation; and multiple sub-matrices can be used to calculate the resume user network and the resume university network, and by clustering the resume user network and the resume university network, the accuracy of the obtained disambiguated resume set can be improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0052] Figure 1 A flowchart of a resume disambiguation method in an embodiment of the present application;
[0053] Figure 2 A schematic diagram of functional modules of a resume disambiguation device provided in one embodiment of the present application;
[0054] Figure 3 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION
[0055] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0056] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.
[0057] In an exemplary embodiment, Figure 1 As shown, a resume disambiguation method is provided, which is executed by a computer device, and can be specifically executed by a computer device such as a terminal or a server alone, or can be executed by a terminal and a server together. In an embodiment of the present application, the method includes the following steps 101 to 108.
[0058] in:
[0059] Step 101, determine the resume information set based on the collected teacher homepage address.
[0060] In an embodiment of the present application, the resume information set includes resume information corresponding to multiple teachers respectively; the resume information includes at least a resume ID, a teacher's name, work information, and education information.
[0061] As an optional implementation, step 101 may specifically include determining the resume information set according to the collected teacher homepage address:
[0062] Obtaining the homepage text information of multiple teachers from the collected teacher homepage addresses;
[0063] Extracting the preset key information in each of the homepage text information to obtain the initial resume information corresponding to each of the homepage text information;
[0064] Preprocess each of the initial resume information to obtain resume information corresponding to each of the initial resume information;
[0065] Integrate each resume information to obtain a resume information set.
[0066] Among them, by implementing this implementation method, by extracting key information and preprocessing the acquired homepage text information, the obtained resume information can be made more unified and standardized.
[0067] In the embodiment of the present application, the homepage address of a university generally has a collection of personal homepages of teachers in the department settings. For example: https: / / tujian.henu.edu.cn / info / 1502 / 5419.htm. And the links of the first ten search results of the homepage address of the university can be crawled through a search engine as the homepage address of the teacher, and the robot.txt protocol is followed during the process.
[0068] In the embodiment of the present application, the homepage text information refers to the teacher's personal homepage text information.
[0069] For example, the main text information of the homepage can be "Li XX, male, from Xiangcheng, Henan, member of the Communist Party of China, master of engineering, experimenter / laborator. Work experience: worked in the laboratory of the School of Civil Engineering and Architecture from September 2004 to present."
[0070] In the embodiment of the present application, the text in the personal homepage of the faculty teacher can be processed by using a web page text extraction algorithm based on logical lines and maximum acceptance distance, and a general web page text extraction algorithm based on block distribution function to obtain the teacher's homepage text information.
[0071] In the embodiment of the present application, the preset key information may include:
[0072] 1. Basic information (resume photo, name, gender, date of birth, mobile phone number, email address, place of origin, time of first employment and expected year of graduation);
[0073] 2. Educational background (school name, education (full-time or not), major, duration of study (start date and month, end date and month, still studying or not));
[0074] 3. Work experience (name of unit, position, employment time (start date and month, end date and month, whether still employed), job content);
[0075] 4. Personal advantages;
[0076] 5. Academic achievements (overview of achievements, detailed description of achievements (papers, patent monographs));
[0077] 6. Project experience (project name, project role, project cycle (start time year month, end time year month, whether still in progress), project description);
[0078] 7. Honorary awards;
[0079] 8. Campus experience (society / organization name, role, time period (starting date and month, ending date and month, whether still in school), description of experience);
[0080] 9. Professional skills;
[0081] 10. Qualification certificate.
[0082] In the embodiment of the present application, the initial resume information can be extracted by a pre-trained extraction model. The extraction model can be trained by selecting training data and test data in advance, and the open source large model can be used to perform Lora fine-tuning to complete the segmentation and disassembly of the initial resume information. Until the accuracy of the model reaches a preset threshold.
[0083] Construct the SFT dataset and the corresponding prompt.
[0084] For example, the SFT dataset looks like:
[0085]
[0086] "value":"-Task: Extract the following information from the text: 1. Basic information 2. Name 3. Gender 4. Date of birth 5. Mobile phone number 6. Email 7. Place of origin 8. Time of first employment 9. Expected year of graduation 2. Educational experience 1. School name 2. Education (whether full-time) 3. Major 4. Time in school (start time year month, end time year month, whether still in progress) 3. Work experience 1. Company name 2. Position name 3. Time in office (start time year month, end time year month, whether still in progress) 4. Work content 4. Personal strengths 5. Academic achievements 1. Overview of achievements 2. Detailed description of achievements (papers, patents and monographs) 6. Project experience 1. Project name 2. Project role 3. Project cycle (opening and closing) Start time (year and month, end time (year and month, still in) 4. Project description 7. Honorary awards 8. Campus experience 1. Club / organization name 2. Role 3. Time period (start time (year and month, end time (year and month, still in) 4. Experience description 9. Professional skills 10. Qualification certificate. -Text: Lei, female, master, lecturer. Graduated from a certain art college. Studied in environmental design at a certain art college. Research direction: watercolor, interior design Main courses: sketching, color. The work has won the Excellence Award of a certain art newcomer new work exhibition; the MOWS Director Award of a certain association; and was selected for an art exhibition. Won the Outstanding Instructor Award at a certain architectural and environmental design student art work grand prize, etc. Participated in the compilation of two textbooks. "
[0087]
[0088] It is equivalent to using a large model to do the NER (named entity recognition) work of small sample training.
[0089] In the embodiment of the present application, according to the project requirements, the extraction model needs to achieve an accuracy rate of 90%. The difficulty here is mainly to ensure that the problem of overfitting does not occur while achieving high accuracy. Therefore, the training set samples will be continuously increased according to the effect of fine-tuning, so that the extraction model can learn how to extract texts with poor text structure or containing certain abnormal data. This part mainly involves manually looking at the wrongly split text through each split result, analyzing the reasons for the wrong split, and then summarizing this type of wrong text structure and adding it to the training set to improve the recognition accuracy.
[0090] In the embodiment of the present application, each initial resume information may be preprocessed by converting uppercase letters to lowercase, removing non-alphabetic characters, removing redundant spaces, segmenting words, removing stop words and short words, etc. In addition, a name dictionary may be constructed to ensure the consistency of names, for example, Yujie Qiu and Qiu Yujie may be unified into qiuyujie.
[0091] Step 102, constructing a resume matrix based on the resume information set.
[0092] In an embodiment of the present application, the resume matrix includes multiple matrix rows and multiple matrix columns, one matrix row corresponds to a resume information, and one matrix column corresponds to a pre-stored university information; the matrix value in the resume matrix represents the correlation between the resume information corresponding to the matrix value and the university information corresponding to the matrix value.
[0093] For example, we can put m resumes in a huge m*n resume matrix. n is all the colleges and universities that have been entered in our database. The matrix value is either 0 or 1, where 1 means teaching at the college or having education experience at the college.
[0094] Step 103, performing singular value decomposition on the resume matrix to obtain multiple sub-matrices.
[0095] In the embodiment of the present application, the multiple sub-matrices include at least a resume user feature matrix, a content feature matrix, and a diagonal matrix. Singular value decomposition (SVD) can decompose the resume matrix into three sub-matrices, specifically:
[0096] A=UΣV^T
[0097] Among them, A represents the resume matrix, U represents the feature matrix of the resume user, Σ represents the diagonal matrix, and V^T represents the content feature matrix. The feature matrix of the resume user is the representation of the resume user in the latent feature space; each row in the feature matrix of the resume user corresponds to a resume user with resume information; the content feature matrix is the representation of the university information in the latent feature space.
[0098] Step 104: Use the multiple sub-matrices to calculate the resume user network and the resume university network.
[0099] As an optional implementation, step 104 uses the multiple sub-matrices to calculate the resume user network and the resume university network in a manner that may include:
[0100] Using the feature matrix of the resume user, a resume user network is calculated;
[0101] Determine a feature space representation of the resume user using the feature matrix of the resume user and the diagonal matrix;
[0102] Determine the feature space representation of the university information using the content feature matrix and the diagonal matrix;
[0103] A resume-university network is determined using the feature space representation of the resume user and the feature space representation of the university information.
[0104] Among them, by implementing this implementation method, the resume user network is directly calculated using the feature matrix of the resume user, which can capture the intrinsic connection between users. Then, the feature matrix of the resume user is spatially represented in combination with the diagonal matrix to further improve the expression accuracy of user features. At the same time, the feature matrix of university information is also processed similarly, so that the characteristics of the university can be accurately presented in a unified space. Finally, by comparing and correlating the two feature space representations, a resume university network is constructed, which effectively reveals the potential matching relationship between users and universities, and provides strong data support for subsequent recommendations or analysis. This method not only improves the accuracy and efficiency of network construction, but also enhances its reliability and practicality in practical applications.
[0105] In the embodiment of the present application, the resume university network represents the correlation between resume users and universities.
[0106] Specifically, using the feature matrix of the resume user, a method of calculating the resume user network may include:
[0107] Based on the feature matrix of the resume users, calculate the cosine similarity between the rows of every two resume users;
[0108] Establish a target edge between two target resume users whose inter-row cosine similarity is greater than a preset threshold;
[0109] Based on the resume users and the target edges, a resume user network is constructed.
[0110] Among them, by implementing this implementation method, by calculating the inter-row cosine similarity between every two users in the resume user feature matrix, the cosine value of the angle between the user feature vectors can be accurately measured, thereby effectively capturing the similarity and correlation between users. Secondly, setting a preset threshold and establishing a target edge between two target resume users whose similarity exceeds the threshold helps to screen out user pairs that are truly strongly correlated and avoid noise interference during the network construction process. Finally, based on resume users and these carefully selected target edges, the resume user network constructed not only has a clear structure and clear node relationships, but also has a high degree of stability and reliability.
[0111] In the embodiment of the present application, the Euclidean distance between the rows of every two resume users can be used instead of the cosine similarity between the rows.
[0112] Step 105: cluster the resume user network and the resume university network to obtain a pre-clustered resume set and an outlier resume set.
[0113] In the embodiment of the present application, a clustering algorithm (such as K-Means, spectral clustering or community detection algorithm) can be used to find resume user groups with similarities, and then obtain a pre-clustered resume set and an outlier resume set.
[0114] In the embodiment of the present application, the resume user network and the resume university network can be first converted into a resume user matrix and a resume university matrix, and the resume user matrix and the resume university matrix are merged and averaged to obtain a final resume similarity matrix. Specifically: the two matrices are weighted and added to obtain the average to obtain the final resume similarity matrix. The weight ratio of the two matrices can be set to 1:1, which is not limited in the embodiment of the present application.
[0115] In the embodiment of the present application, the DBSCAN algorithm can be used to perform preliminary clustering on the resume similarity matrix to obtain a pre-clustered resume set and an outlier resume set. The pre-clustered resume set includes multiple pre-clustered resume clusters, each pre-clustered resume cluster contains multiple pre-clustered resumes, and the outlier resume combination includes an outlier resume.
[0116] Step 106: Based on the pre-clustered resume set, the outlier resume set is integrated to obtain a disambiguated resume set.
[0117] As an optional implementation, step 108 integrates the outlier resume set based on the pre-clustered resume set to obtain the disambiguated resume set, which may specifically include:
[0118] Acquire a target outlier resume from the outlier resume set;
[0119] Calculate and obtain a first similarity between the target outlier resume and each pre-clustered resume;
[0120] If the maximum first similarity is greater than or equal to a preset threshold, the target outlier resume is moved from the outlier resume set to the pre-clustered resume cluster where the target pre-clustered resume corresponding to the maximum first similarity is located;
[0121] If the maximum first similarity is less than the preset threshold, constructing a new pre-clustered resume cluster in the pre-clustered resume set, and moving the target outlier resume from the outlier resume set to the new pre-clustered resume cluster;
[0122] Calculate and obtain a second similarity between the target outlier resume and each current outlier resume in the outlier resume set;
[0123] Determine a target current outlier resume from the outlier resume set; wherein the second similarity corresponding to the target current outlier resume is greater than or equal to the preset threshold;
[0124] Move the target current outlier resume from the outlier resume set to the pre-clustered resume cluster where the target outlier resume is located;
[0125] If there are still outlier resumes in the outlier resume set, executing the step from obtaining the target outlier resume from the outlier resume set to moving the target current outlier resume from the outlier resume set to the pre-clustered resume cluster where the target outlier resume is located;
[0126] If there is no outlier resume in the outlier resume set, the current pre-clustered resume set is determined as the disambiguation resume set.
[0127] Among them, by implementing this implementation method, feature matching and similarity calculation can be performed on outlier resumes, and they can be reallocated to corresponding clusters to improve the overall clustering effect and the robustness of the disambiguation results.
[0128] In the embodiment of the present application, the first similarity between the target outlier resume and a pre-clustered resume can be calculated as follows:
[0129] The first similarity s(pi,pj) of two resumes pi and pj is defined as follows:
[0130] 1) Initially s(pi,pj) is 0;
[0131] 2) s(pi,pj) = s(pi,pj) + (the number of common educational units of pi and pj) × 1.5;
[0132] 3) s(pi,pj)=s(pi,pj)+tanimoto(pi’s mail, pj’s mail);
[0133] 4) s(pi,pj) = s(pi,pj) + tanimoto (the working unit of the name to be disambiguated in pi, the working unit of the name to be disambiguated in pj);
[0134] 5) s(pi,pj) = s(pi,pj) + (number of common words in title in pi and pj) / 3.0;
[0135] 6) Output s(pi,pj);
[0136] Where tanimoto(p,q) refers to the tanimoto similarity of two sequences, which is defined as follows:
[0137] def tanimoto(p,q):
[0138] c=[v for v in p if v in q]
[0139] return float(len(c) / (len(p)+len(q)-len(c)))
[0140] In the embodiment of the present application, the first similarity between resumes includes mail similarity, education / work organization similarity, number of common words in the title, etc., which is not limited in the embodiment of the present application.
[0141] In the embodiment of the present application, the calculation method of the second similarity between the target outlier resume and a current outlier resume in the outlier resume set is the same as the calculation method of the first similarity, which will not be repeated here.
[0142] In the embodiment of the present application, the reason for generating and matching outlier resumes is that the features of these resumes are often not obvious enough, resulting in a relatively small similarity between them and other resumes, or these resumes themselves belong to a person with less educational experience. The effect of using the representation vector learning method mentioned above for these forms of resumes is not as good as the effect of using the method of directly using features for matching.
[0143] After the above operations, the resumes in the outlier resume set are also assigned to their respective clusters, and the outlier resume set and the pre-clustered resume set are integrated, so that the disambiguated resume set of all resumes is obtained. This disambiguated resume set is our final disambiguation result for a certain name to be disambiguated.
[0144] In addition, the resume information of the candidate can be obtained from the disambiguation resume collection, and the candidate-resume association result can be output. The disambiguation resume collection can also be updated regularly to reflect the latest candidate information and resume data.
[0145] As an optional implementation, the method for obtaining the disambiguation resume set may also be:
[0146] 1. Determine the resume information set based on the collected teacher homepage addresses.
[0147] 2. Construct a resume heterogeneous network based on the resume information set.
[0148] In an embodiment of the present application, the resume heterogeneous network includes multiple nodes, one resume information corresponds to one node; the edge between two nodes is constructed based on the same work information and / or the same education information in the resume information corresponding to the two nodes.
[0149] 3. Based on the resume heterogeneous network, multiple resume ID paths are obtained through random walks of preset meta-paths.
[0150] In the embodiment of the present application, the preset meta-path may be a path pattern such as P1-CoEdu-P2-CoWork-P3. During the random walk, the walk may be performed in the order of the preset meta-paths.
[0151] During each walk, the next node is selected according to a certain type of edge specified by the current meta-path, and a node connected to the current node through this type of edge is randomly selected as the next node. In each long path, such meta-paths are repeatedly sampled several times, that is, the last node of the previous meta-path is used as the first node of the next meta-path.
[0152] Finally, iterate until a certain number of times (defined as walklength in the code) is reached, then another node is selected as the starting node for walking, and finally several such long paths are generated. Each path point is the ID of the resume, and each long path is stored by row.
[0153] At the same time, when a node randomly selects the next node to walk towards a certain type of edge under the guidance of the meta-path, the weight of the edge will be taken into consideration. The larger the weight, the closer the relationship between the two nodes, and the greater the probability of the node jumping along this relationship. We stipulate that the probability is in direct proportion to the weight.
[0154] In some cases, some relationships are missing for some resumes. For example, all the educational units in a resume do not appear in the list of educational units of any other resume. In this case, the CoEdu relationship is missing. When this happens, a more flexible strategy is adopted, that is, to roam according to the next relationship of the current missing relationship in the meta-path. For the resume mentioned above, it will roam according to its CoWork relationship instead.
[0155] In an embodiment of the present application, the generation of a resume ID path can be achieved through a pre-trained path walking model.
[0156] 4. Based on multiple resume ID paths, the resume relationship similarity matrix is calculated.
[0157] In the embodiment of the present application, the skip-gram model of word2vec in the gensim library can be used to process multiple resume ID paths, and the multiple resume ID paths are regarded as a corpus, and the resume IDs in the corpus are regarded as words, and finally the representation vector corresponding to each resume ID is obtained. Since what is actually stored in this corpus is the relationship between the common education and work unit name similarity between nodes, we call this representation vector a node relationship representation vector.
[0158] The node relationship representation vector is expressed as a matrix of resume relationship similarity.
[0159] As an optional implementation, based on multiple resume ID paths, the method of calculating the resume relationship similarity matrix may specifically include:
[0160] Calculate each resume ID path to obtain the cosine similarity matrix of each resume ID path;
[0161] The mean of multiple cosine similarity matrices is calculated to obtain the resume relationship similarity matrix.
[0162] Among them, by implementing this implementation method, the resume relationship similarity matrix can be obtained by calculating the resume ID path, thereby capturing the complex relationship information between resumes and improving the accuracy of homonym disambiguation.
[0163] In the embodiment of the present application, we do not save the starting point in the resume ID path, so for isolated nodes in the graph (i.e. nodes that have no edges connected to other nodes), the resume ID corresponding to this node does not exist in the path set, and its node relationship representation vector cannot be obtained. The relationship representation vectors of these nodes can be set to all 0 vectors and saved in the outlier resume set for subsequent processing.
[0164] When training the word2vec model, you can use the idea of Bagging, adopt k path sets, use k resume ID paths to train k word2vec models, generate k groups of resume vectors, calculate the cosine similarity matrix for each group of resume vectors, and then calculate the average of these k similarity matrices to get the final resume relationship similarity matrix.
[0165] 5. Merge the multiple resume information contained in the resume information set to obtain the resume text corresponding to each resume information.
[0166] As an optional implementation, a method of merging the multiple resume information included in the resume information set to obtain the resume text corresponding to each resume information may specifically include:
[0167] Obtaining preset parameters of each resume information from the resume information set; wherein the preset parameters include gender information, place of origin information, date of birth, job title information, and email information;
[0168] Merge the preset parameters of each resume information to obtain the initial resume text corresponding to each resume information;
[0169] Preprocess each initial resume text to obtain a resume text corresponding to each initial resume text.
[0170] Among them, by implementing this implementation method, the preset parameters of each resume information obtained can be merged, and each initial resume text obtained by the merger can be preprocessed, so that the format of the obtained resume texts is a unified format, and the content of the resume texts does not contain meaningless content, thereby ensuring the uniformity of the resume texts.
[0171] In the embodiment of the present application, the preset parameters may include name, gender, birthplace, date of birth, job title, email address, education, work experience, etc., which are not limited in the embodiment of the present application. The preset parameters of each resume information are combined by separating name, title, education, work, award, and mail with spaces to form an initial resume text.
[0172] In the embodiment of the present application, each initial resume text is preprocessed by converting uppercase letters to lowercase, removing non-alphabetic characters, removing redundant spaces, and using spaces to segment words to remove stop words and words with a length of less than 3.
[0173] 6. Process multiple resume texts to obtain a resume semantic similarity matrix.
[0174] In the embodiment of the present application, multiple resume texts can be processed by a pre-trained gensim word2vec model to obtain a resume semantic similarity matrix. The process of training the word2vec model can be: extracting the texts in the training set, the validation set, and the test set into a text corpus.
[0175] Specifically, for each resume, we extracted the text of each resume author's unit, name, email address, and research field, and stored them in a txt file line by line. When the test set was not available, we only used the text of the training set and validation set. Then we used this txt file as the corpus and used gensim's word2vec model to train word vectors.
[0176] As an optional implementation, a method of processing multiple resume texts to obtain a resume semantic similarity matrix may specifically include:
[0177] Process multiple resume texts separately to obtain the word vectors of each resume text;
[0178] Calculate the mean of the word vectors of each resume text to obtain the semantic representation vector of each resume text;
[0179] The cosine similarity is calculated for the semantic representation vectors of each resume text to obtain the resume semantic similarity matrix.
[0180] Among them, this implementation method is implemented by processing the resume text to obtain word vectors, and then the word vectors are averaged to obtain the semantic representation vector of the resume text, and then the cosine similarity of the semantic representation vector is calculated to obtain the resume semantic similarity matrix, which can accurately capture the semantic information in the text and provide high-quality input for subsequent clustering.
[0181] In the embodiment of the present application, word vectors are generated and averaged using the word2vec model trained in the previous paragraph, ultimately obtaining a semantic representation vector for the resume text.
[0182] For each resume name to be disambiguated, the semantic representation vectors of all its resumes are obtained. When all the words of a resume do not exist in the word2vec model, its semantic representation vector is set to all 0 and saved in the outlier resume set for further processing. Finally, the cosine similarity between the resumes is calculated to obtain the resume semantic similarity matrix.
[0183] 7. Cluster the resume relationship similarity matrix and the resume semantic similarity matrix to obtain the pre-clustered resume set and the outlier resume set.
[0184] In the embodiment of the present application, the resume relationship similarity matrix and the resume semantic similarity matrix can be merged and averaged to obtain the final resume similarity matrix. Specifically: the two matrices are weighted and added to obtain the average to obtain the final resume similarity matrix. The weight ratio of the two matrices can be set to 1:1, which is not limited in the embodiment of the present application.
[0185] In the embodiment of the present application, the DBSCAN algorithm can be used to perform preliminary clustering on the resume similarity matrix to obtain a pre-clustered resume set and an outlier resume set. The pre-clustered resume set includes multiple pre-clustered resume clusters, each pre-clustered resume cluster contains multiple pre-clustered resumes, and the outlier resume combination includes outlier resumes.
[0186] The result of clustering is generally speaking which resumes belong to which person. For example, if there are two teachers named Zhang San, all the resumes of Zhang San A are clustered into cluster 1, and all the resumes of Zhang San B are clustered into cluster 2. And so on.
[0187] 8. Based on the pre-clustered resume set, the outlier resume set is integrated to obtain the disambiguated resume set.
[0188] By implementing the above-mentioned steps 101 to 106, it is possible to capture the complex relationship information between resumes and improve the accuracy of homonymous disambiguation; and multiple submatrices can be used to calculate the resume user network and the resume university network, and by clustering the resume user network and the resume university network, the accuracy of the obtained disambiguated resume set can be improved. In addition, the present application can also make the obtained resume information more unified and standardized. In addition, the present application can also improve the accuracy and efficiency of the construction of the resume user network and the resume university network. In addition, the present application can also construct a resume user network with a clear structure and clear node relationships. In addition, the present application can also improve the overall clustering effect and the robustness of the disambiguation results.
[0189] The present application also provides an application scenario, which applies the above-mentioned resume disambiguation method. Specifically: The resume disambiguation method provided in this embodiment can be applied in a resume screening scenario. In the process of obtaining the resume information of the target teacher, the accurate resume information of the target teacher can be obtained from the resume set that has been disambiguated. The resume disambiguation method provided in this embodiment belongs to the link of disambiguating the resume set.
[0190] Based on the same inventive concept, the embodiment of the present application also provides a resume disambiguation device for implementing the resume disambiguation method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more resume disambiguation device embodiments provided below can refer to the limitations of the resume disambiguation method above, and will not be repeated here.
[0191] In an exemplary embodiment, Figure 2 As shown, a resume disambiguation device is provided, comprising:
[0192] The determining unit 201 is used to determine a resume information set according to the collected teacher homepage address; wherein the resume information set includes resume information corresponding to multiple teachers respectively; the resume information at least includes a resume ID, a teacher's name, work information, and education information;
[0193] A construction unit 202 is used to construct a resume matrix according to the resume information set; wherein the resume matrix includes a plurality of matrix rows and a plurality of matrix columns, one matrix row corresponds to one resume information, and one matrix column corresponds to one pre-stored university information; the matrix value in the resume matrix represents the correlation between the resume information corresponding to the matrix value and the university information corresponding to the matrix value;
[0194] The decomposition unit 203 is used to perform singular value decomposition on the resume matrix to obtain multiple sub-matrices; wherein the multiple sub-matrices at least include a resume user's feature matrix, a content feature matrix, and a diagonal matrix; the resume user's feature matrix is a representation of the resume user in a potential feature space; each row in the resume user's feature matrix corresponds to a resume user of resume information; the content feature matrix is a representation of the university information in the potential feature space;
[0195] A calculation unit 204 is used to calculate the resume user network and the resume university network using the multiple sub-matrices;
[0196] A clustering unit 205, used for clustering the resume user network and the resume university network to obtain a pre-clustered resume set and an outlier resume set;
[0197] The integration unit 206 is used to integrate the outlier resume set based on the pre-clustered resume set to obtain a disambiguated resume set.
[0198] As an optional implementation, the determination unit 201 may determine the resume information set according to the collected teacher homepage address in the following manner:
[0199] Obtaining the homepage text information of multiple teachers from the collected teacher homepage addresses;
[0200] Extracting the preset key information in each of the homepage text information to obtain the initial resume information corresponding to each of the homepage text information;
[0201] Preprocess each of the initial resume information to obtain resume information corresponding to each of the initial resume information;
[0202] Integrate each resume information to obtain a resume information set.
[0203] Among them, by implementing this implementation method, by extracting key information and preprocessing the acquired homepage text information, the obtained resume information can be made more unified and standardized.
[0204] As an optional implementation, the calculation unit 204 uses the multiple sub-matrices to calculate the resume user network and the resume university network in the following manner:
[0205] Using the feature matrix of the resume user, a resume user network is calculated;
[0206] Determine a feature space representation of the resume user using the feature matrix of the resume user and the diagonal matrix;
[0207] Determine the feature space representation of the university information using the content feature matrix and the diagonal matrix;
[0208] A resume-university network is determined using the feature space representation of the resume user and the feature space representation of the university information.
[0209] Among them, by implementing this implementation method, the resume user network is directly calculated using the feature matrix of the resume user, which can capture the intrinsic connection between users. Then, the feature matrix of the resume user is spatially represented in combination with the diagonal matrix to further improve the expression accuracy of user features. At the same time, the feature matrix of university information is also processed similarly, so that the characteristics of the university can be accurately presented in a unified space. Finally, by comparing and correlating the two feature space representations, a resume university network is constructed, which effectively reveals the potential matching relationship between users and universities, and provides strong data support for subsequent recommendations or analysis. This method not only improves the accuracy and efficiency of network construction, but also enhances its reliability and practicality in practical applications.
[0210] As an optional implementation, the calculation unit 204 uses the feature matrix of the resume user to calculate the resume user network in the following manner:
[0211] Based on the feature matrix of the resume users, calculate the cosine similarity between the rows of every two resume users;
[0212] Establish a target edge between two target resume users whose inter-row cosine similarity is greater than a preset threshold;
[0213] Based on the resume users and the target edges, a resume user network is constructed.
[0214] Among them, by implementing this implementation method, by calculating the inter-row cosine similarity between every two users in the resume user feature matrix, the cosine value of the angle between the user feature vectors can be accurately measured, thereby effectively capturing the similarity and correlation between users. Secondly, setting a preset threshold and establishing a target edge between two target resume users whose similarity exceeds the threshold helps to screen out user pairs that are truly strongly correlated and avoid noise interference during the network construction process. Finally, based on resume users and these carefully selected target edges, the resume user network constructed not only has a clear structure and clear node relationships, but also has a high degree of stability and reliability.
[0215] As an optional implementation, the pre-clustered resume set includes multiple pre-clustered resume clusters, each pre-clustered resume cluster includes multiple pre-clustered resumes, the outlier resume combination includes outlier resumes, and the integration unit 206 integrates the outlier resume set based on the pre-clustered resume set to obtain the disambiguated resume set in the following manner:
[0216] Acquire a target outlier resume from the outlier resume set;
[0217] Calculate and obtain a first similarity between the target outlier resume and each pre-clustered resume;
[0218] If the maximum first similarity is greater than or equal to a preset threshold, the target outlier resume is moved from the outlier resume set to the pre-clustered resume cluster where the target pre-clustered resume corresponding to the maximum first similarity is located;
[0219] If the maximum first similarity is less than the preset threshold, constructing a new pre-clustered resume cluster in the pre-clustered resume set, and moving the target outlier resume from the outlier resume set to the new pre-clustered resume cluster;
[0220] Calculate and obtain a second similarity between the target outlier resume and each current outlier resume in the outlier resume set;
[0221] Determine a target current outlier resume from the outlier resume set; wherein the second similarity corresponding to the target current outlier resume is greater than or equal to the preset threshold;
[0222] Move the target current outlier resume from the outlier resume set to the pre-clustered resume cluster where the target outlier resume is located;
[0223] If there are still outlier resumes in the outlier resume set, executing the step from obtaining the target outlier resume from the outlier resume set to moving the target current outlier resume from the outlier resume set to the pre-clustered resume cluster where the target outlier resume is located;
[0224] If there is no outlier resume in the outlier resume set, the current pre-clustered resume set is determined as the disambiguation resume set.
[0225] Among them, by implementing this implementation method, feature matching and similarity calculation can be performed on outlier resumes, and they can be reallocated to corresponding clusters to improve the overall clustering effect and the robustness of the disambiguation results.
[0226] By implementing the above-mentioned implementation mode, it is possible to capture the complex relationship information between resumes and improve the accuracy of homonymous disambiguation; and multiple submatrices can be used to calculate the resume user network and the resume university network, and by clustering the resume user network and the resume university network, the accuracy of the obtained disambiguated resume set can be improved. In addition, the present application can also make the obtained resume information more unified and standardized. In addition, the present application can also improve the accuracy and efficiency of the construction of the resume user network and the resume university network. In addition, the present application can also construct a resume user network with a clear structure and clear node relationships. In addition, the present application can also improve the overall clustering effect and the robustness of the disambiguation results.
[0227] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store resume disambiguation data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a resume disambiguation method is implemented.
[0228] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0229] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.
[0230] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.
[0231] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.
[0232] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.
[0233] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).
[0234] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.
[0235] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0236] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.
Claims
1. A resume disambiguation method, characterized in that: The resume disambiguation method includes: Determine a resume information set according to the collected teacher homepage address; wherein the resume information set includes resume information corresponding to multiple teachers respectively; the resume information includes at least a resume ID, a teacher's name, work information, and education information; Constructing a resume matrix according to the resume information set; wherein the resume matrix includes a plurality of matrix rows and a plurality of matrix columns, one matrix row corresponds to one resume information, and one matrix column corresponds to one pre-stored university information; the matrix value in the resume matrix represents the correlation between the resume information corresponding to the matrix value and the university information corresponding to the matrix value; Performing singular value decomposition on the resume matrix to obtain multiple sub-matrices; wherein the multiple sub-matrices at least include a resume user's feature matrix, a content feature matrix, and a diagonal matrix; Using the multiple sub-matrices, a resume user network and a resume university network are calculated; Clustering the resume user network and the resume university network to obtain a pre-clustered resume set and an outlier resume set; Based on the pre-clustered resume set, the outlier resume set is integrated to obtain a disambiguated resume set.
2. The resume disambiguation method according to claim 1, characterized in that: Determining the resume information set based on the collected teacher homepage address specifically includes: Obtaining the homepage text information of multiple teachers from the collected teacher homepage addresses; Extracting the preset key information in each of the homepage text information to obtain the initial resume information corresponding to each of the homepage text information; Preprocess each of the initial resume information to obtain resume information corresponding to each of the initial resume information; Integrate each resume information to obtain a resume information set.
3. The resume disambiguation method according to claim 1, characterized in that: in: The feature matrix of the resume user is a representation of the resume user in the potential feature space; each row in the feature matrix of the resume user corresponds to a resume user with resume information; The content feature matrix is a representation of the university information in the potential feature space.
4. The resume disambiguation method according to claim 3, characterized in that: The use of the multiple sub-matrices to calculate the resume user network and the resume university network specifically includes: Using the feature matrix of the resume user, a resume user network is calculated; Determine a feature space representation of the resume user using the feature matrix of the resume user and the diagonal matrix; Determine the feature space representation of the university information using the content feature matrix and the diagonal matrix; A resume-university network is determined using the feature space representation of the resume user and the feature space representation of the university information.
5. The resume disambiguation method according to claim 4, characterized in that: The use of the feature matrix of the resume user to calculate the resume user network specifically includes: Based on the feature matrix of the resume users, calculate the cosine similarity between the rows of every two resume users; Establish a target edge between two target resume users whose inter-row cosine similarity is greater than a preset threshold; Based on the resume users and the target edges, a resume user network is constructed.
6. The resume disambiguation method according to any one of claims 1 to 5, characterized in that: The pre-clustered resume set includes multiple pre-clustered resume clusters, each pre-clustered resume cluster includes multiple pre-clustered resumes, the outlier resume combination includes outlier resumes, and the outlier resume set is integrated based on the pre-clustered resume set to obtain the disambiguated resume set, specifically including: Acquire a target outlier resume from the outlier resume set; Calculate and obtain a first similarity between the target outlier resume and each pre-clustered resume; If the maximum first similarity is greater than or equal to a preset threshold, the target outlier resume is moved from the outlier resume set to the pre-clustered resume cluster where the target pre-clustered resume corresponding to the maximum first similarity is located; If the maximum first similarity is less than the preset threshold, constructing a new pre-clustered resume cluster in the pre-clustered resume set, and moving the target outlier resume from the outlier resume set to the new pre-clustered resume cluster; Calculate and obtain a second similarity between the target outlier resume and each current outlier resume in the outlier resume set; Determine a target current outlier resume from the outlier resume set; wherein the second similarity corresponding to the target current outlier resume is greater than or equal to the preset threshold; Move the target current outlier resume from the outlier resume set to the pre-clustered resume cluster where the target outlier resume is located; If there are still outlier resumes in the outlier resume set, executing the step from obtaining the target outlier resume from the outlier resume set to moving the target current outlier resume from the outlier resume set to the pre-clustered resume cluster where the target outlier resume is located; If there is no outlier resume in the outlier resume set, the current pre-clustered resume set is determined as the disambiguation resume set.
7. A resume disambiguation device, characterized in that: The resume disambiguation device comprises: A determination unit, configured to determine a resume information set according to the collected teacher homepage address; wherein the resume information set includes resume information corresponding to a plurality of teachers respectively; and the resume information includes at least a resume ID, a teacher's name, work information, and education information; A construction unit is used to construct a resume matrix according to the resume information set; wherein the resume matrix includes a plurality of matrix rows and a plurality of matrix columns, one matrix row corresponds to one resume information, and one matrix column corresponds to one pre-stored university information; the matrix value in the resume matrix represents the correlation between the resume information corresponding to the matrix value and the university information corresponding to the matrix value; A decomposition unit, used to perform singular value decomposition on the resume matrix to obtain multiple sub-matrices; wherein the multiple sub-matrices at least include a feature matrix of the resume user, a content feature matrix, and a diagonal matrix; A calculation unit, used to calculate the resume user network and the resume university network using the multiple sub-matrices; A clustering unit, used for clustering the resume user network and the resume university network to obtain a pre-clustered resume set and an outlier resume set; An integration unit is used to integrate the outlier resume set based on the pre-clustered resume set to obtain a disambiguated resume set.
8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the resume disambiguation method described in any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the resume disambiguation method described in any one of claims 1 to 6 are implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the resume disambiguation method described in any one of claims 1 to 6 are implemented.