Post recommendation method and device based on big data and storage medium

Abstract

The invention relates to the technical field of post recommendation, in particular to a post recommendation method and device based on big data and a storage medium, and the method comprises the steps: obtaining original post information, extracting a plurality of original post features from the original post information, and forming an original post feature set; judging whether the demand features are matched with the basic features and the capability features or not to form a prejudice feature set; eliminating unmatched original post features from the original post feature set to form a missing post feature set; on the basis of the missing post feature set, searching new post features matched with elements in the missing post feature set from a database or a network; storing the new post features in the missing post feature set to form a new post feature set, and repairing the original post information according to the new post feature set to obtain new post information; and searching to-be-recommended personnel information matched with the new post feature set in a database by using a pre-stored post recommendation model.

Description

Technical Field

[0001] The present invention relates to the technical field of job recommendation, and in particular to a job recommendation method, device and storage medium based on big data. Background Art

[0002] With the development of big data and Internet technology, during the process of corporate recruitment and job seekers' job hunting, the job recommendation system of each job search website will match keywords based on the job information in the company's recruitment and the job seekers' resumes, and judge whether the matching degree meets the recommendation requirements based on the set threshold, and then automatically push the company's job information to the matching job seekers to ensure the efficiency of corporate recruitment; however, in the recruitment process, various job search requirements set by the recruiting companies are often encountered, some of which are not fair, making it difficult for some excellent job seekers to receive job push information from the corresponding companies. Summary of the invention

[0003] The present invention extracts original job features from job information released by enterprises, identifies bias features from the original job features, obtains new job features that can replace bias features through big data analysis, replaces bias features with the obtained new job features, identifies the matching degree of recommended personnel who match the new job features, monitors the job recommendation model in real time, reduces the algorithm bias of the job recommendation model, and thus ensures the fairness of job recommendations.

[0004] The technical solution proposed by the present invention is: a job recommendation method based on big data, the method comprising:

[0005] Acquire original job information, extract multiple original job features from the original job information to form an original job feature set; the original job information includes one or more of age, gender, ethnicity, education, work experience, skill level and salary;

[0006] Classify the elements in the original job feature set into basic features, ability features and demand features, judge whether the demand features match the basic features and ability features, conduct bias evaluation on the original job feature set based on the judgment results, and extract the unmatched original job feature information to form a bias feature set;

[0007] Excluding unmatched original job features from the original job feature set to form a missing job feature set; based on the missing job feature set, searching for new job features that match the elements in the missing job feature set from a database or network;

[0008] The new position features are stored in the missing position feature set to form a new position feature set, and the original position information is repaired according to the new position feature set to obtain the new position information;

[0009] Use the pre-stored job recommendation model to find the recommended personnel information in the database that matches the new job feature set. Based on the output of the job recommendation model, use the fairness index to evaluate the fairness of the job recommendation model to determine whether the job recommendation model has algorithmic bias. If so, adjust the parameters of the job recommendation model and re-match the recommended personnel. Otherwise, proceed to the next step.

[0010] The repaired original job information is recommended to the recommended personnel, and the repair record is fed back to the host computer.

[0011] Preferably, the maximum age, gender, and nationality characteristics are extracted from the original job information to form a basic feature set; the highest education, minimum years of work experience, and minimum skill level characteristics are extracted to form a capability feature set; the highest salary, work content feature vector, and job responsibility feature vector are extracted to form a demand feature set;

[0012] The extraction of the highest salary, work content feature vector and position responsibility feature vector to form a demand feature set includes:

[0013] Use TF-IDF vectorization method to convert the job content description in the original job information into a job content feature vector;

[0014] Use TF-IDF vectorization method to convert the job description in the original job information into a job feature vector;

[0015] The extracted highest salary, work content feature vector and position responsibility feature vector constitute the demand feature set.

[0016] Preferably, the determining whether the demand characteristics match the basic characteristics and the capability characteristics includes:

[0017] Constructing feature relationship matrix ;in, Represent the eigenvalues ​​of the highest age, gender, and ethnicity information, respectively. They represent the eigenvalues ​​of the minimum years of work, the minimum skill level, and the highest education level, respectively. They represent the modulus of the job content feature vector, the eigenvalue of the minimum salary feature, and the modulus of the job responsibility feature vector respectively;

[0018] The row vector of the feature relationship matrix is ​​extracted as the input vector, and after preprocessing, it is input into the pre-trained random forest model to determine whether the demand characteristics match the corresponding basic characteristics and capability characteristics, and output the judgment result.

[0019] Preferably, the bias evaluation is performed on the original job feature set according to the judgment result, and the unmatched original job feature information is extracted to form a bias feature set, including:

[0020] Get the judgment result. If the required features do not match the corresponding basic features and capability features, find the unmatched feature pairs. or ,in Respectively represent the feature value of any basic feature in the basic feature set, capability feature set and requirement feature set;

[0021] The method of finding unmatched feature pairs comprises the following steps:

[0022] The row vector in the feature relationship matrix is ​​represented as:

[0023] ,but The feature pairs included in are: ;in, , represents the number of rows of the feature relationship matrix, ;

[0024] Calculating distance ;

[0025] if Greater than the preset distance threshold , then traverse each feature pair and calculate the distance after removing the corresponding feature pair ,if Less than , then the corresponding feature pair is judged to lead to If the distance is greater than a preset threshold, the demand feature does not match the corresponding basic feature and capability feature; the corresponding feature pair is a biased feature pair;

[0026] The original job feature information corresponding to the corresponding feature pairs is obtained to form a bias feature set.

[0027] Preferably, the step of excluding unmatched original job features from the original job feature set to form a missing job feature set includes:

[0028] Obtaining demand characteristics and / or basic characteristics, and capability characteristics corresponding to the bias characteristics;

[0029] Remove the demand characteristics and / or basic characteristics, and ability characteristics corresponding to the biased characteristics from the original job characteristic set to form a missing job set;

[0030] Construct the feature relationship matrix corresponding to the missing job set ;

[0031] The eigenvalue or mode corresponding to the bias feature pair is The value of the corresponding position in is set to 0.

[0032] Preferably, the method of searching for new job features matching the elements in the missing job feature set from a database or a network based on the missing job feature set comprises the following steps:

[0033] Get feature relationship matrix ,

[0034] Based on the missing job feature set, crawl the job information corresponding to the missing job feature set from the network;

[0035] Use TF-IDF vectorizer to vectorize the crawled job description information and convert it into job content feature vector and job responsibility feature vector;

[0036] Use the K-means algorithm to cluster the work content feature vectors and job responsibility feature vectors, calculate the proportion of bias features in each cluster, and find the cluster with the largest proportion;

[0037] Extracting new job features corresponding to the bias feature pair from the cluster with the largest proportion, obtaining the corresponding new job feature value and / or the modulus of the new feature vector to form a new job feature pair; the modulus of the new job feature value or the new feature vector is different from the value of the element in the bias feature pair;

[0038] Modular replacement using new position characteristic values ​​and new characteristic lines The 0 value at the corresponding position in ;

[0039] The storing of the new position features into the missing position feature set to form a new position feature set, and repairing the original position information according to the new position feature set, includes:

[0040] Obtain new position feature values ​​and new feature vectors;

[0041] Use the new position characteristic values ​​to replace the values ​​of the position characteristics in the original position characteristic set;

[0042] Use the new feature vector to replace the position feature vector in the original position feature set.

[0043] Preferably, the method of using a pre-stored job recommendation model to search for information of personnel to be recommended that matches the new job feature set in a database includes:

[0044] Get the recommended personnel information in the database and extract the personnel resume information from the personnel information;

[0045] Use TF-IDF vectorizer to vectorize the job information of the resume to obtain a job position vector, where the job position vector includes the basic characteristics, ability characteristics and demand characteristics of the job seeker;

[0046] The job vector is input into the job recommendation model, and the job recommendation model combines the job vector with The row vectors in are compared and the similarity is calculated; if the similarity is greater than the recommendation threshold, it is judged that the recommended person information matches the new position feature set, otherwise, it is judged that the recommended person information does not match the new position feature set; finally, the matching result is output.

[0047] Preferably, the output result based on the job recommendation model uses a fairness index to evaluate the fairness of the job recommendation model to determine whether the job recommendation model has algorithmic bias; if so, adjust the parameters of the job recommendation model and re-match the recommended personnel, including:

[0048] Obtain a new job feature set and check whether there is an unbalanced distribution of gender, ethnicity, age, and education level in the new job feature set to prevent one or more features from dominating the new job feature set.

[0049] Using the original job feature set, recommending job information to the recommended person through the job recommendation model;

[0050] Using the new job feature set, recommend job information to the recommended personnel through the job recommendation model;

[0051] Use mean square error (MSE) and absolute error (MAE) as fairness indicators to calculate the prediction errors of the original job feature set and the new job feature set; specifically:

[0052] Calculate the MSE and MAS of the job prediction model under the original job feature set and the new job feature set, and compare the difference values ​​of the fairness index on the original job feature set and the new job feature set;

[0053] If the difference between MSE and MAS on the new job feature set is smaller than that on the original job feature set, the new job feature set is judged to be fairer than the original job feature set;

[0054] Otherwise, adjusting the parameters of the job recommendation model; the adjusting the parameters of the job recommendation model includes one or more of hyperparameter tuning, adjusting the linear regression coefficient, and adjusting the number of layers of the neural network;

[0055] The process of recommending the repaired original job information to the recommended personnel and feeding back the repair record to the host computer includes:

[0056] Push the new job information corresponding to the new job information set to the matching recommended personnel;

[0057] Feedback the job characteristics corresponding to the bias characteristics to the host computer;

[0058] Feedback the new position information set to the host computer for management personnel to query.

[0059] The present invention also provides a job recommendation device based on big data, including a processor, a memory connected to the processor and a communication module, wherein the processor is connected to an external host computer through the communication module, and the device is used to execute the job recommendation method based on big data.

[0060] The present invention also provides a readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the job recommendation method based on big data.

[0061] Beneficial effects of the present invention:

[0062] 1. In order to ensure the fairness of job recommendations, the present invention actively performs bias analysis on original job features, identifies bias features in the original job features, and avoids inputting job features with bias features into the job recommendation model, which affects the fairness of the recommendations of the job recommendation model.

[0063] 2. After identifying biased features, the present invention actively searches for reasonable job features to replace the original job features, modifies the original job information, and searches for matching personnel to be recommended based on the modified new job information; at the same time, when the job recommendation model makes job recommendations, the recommendation process of the job recommendation model is monitored to determine whether there is algorithmic bias in the job recommendation model, which can further ensure the fairness of job recommendations. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 The present invention is a flowchart of a job recommendation method based on big data. DETAILED DESCRIPTION

[0065] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art can think of other obvious variations. The basic principles of the present invention defined in the following description can be applied to other embodiments, variations, improvements, equivalents, and other technical solutions that do not deviate from the spirit and scope of the present invention.

[0066] It is to be understood that the term "one" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element may be one, while in another embodiment, the number of the element may be multiple, and the term "one" should not be understood as a limitation on the quantity.

[0067] refer to Figure 1 The technical solution provided by the present invention is: a job recommendation method based on big data, comprising the following steps:

[0068] 1. Obtaining original job information, extracting multiple original job features from the original job information to form an original job feature set; the original job information includes one or more of age, gender, ethnicity, education, work experience, skill level and salary;

[0069] Then, the maximum age, gender, and ethnicity characteristics are extracted from the original job information to form the basic feature set; the highest education, minimum working years, and minimum skill level characteristics are extracted to form the ability feature set; the highest salary, work content feature vector, and job responsibility feature vector are extracted to form the demand feature set; among them, the maximum age, minimum working years, minimum skill data and other characteristics can be directly expressed in digital form; gender can be quantified using 0 and 1, for example, 0 represents male and 1 represents female; ethnicity can be quantified using one-hot encoding, and the ability characteristics are positive values ​​generated by the system according to the minimum working years and minimum skill data in accordance with the enterprise skill assessment rules. The larger the value, the stronger the ability.

[0070] The extraction of the highest salary, work content feature vector and position responsibility feature vector to form a demand feature set includes:

[0071] The TF-IDF vectorization method is used to convert the job content description and job responsibility description in the original job information into a job content feature vector and a job responsibility feature vector; the extracted maximum salary, job content feature vector and job responsibility feature vector constitute a demand feature set. The maximum salary is directly stored in the demand feature set in digital format.

[0072] Second, classify the elements in the original job feature set into basic features, ability features and demand features, determine whether the demand features match the basic features and ability features, conduct bias evaluation on the original job feature set based on the judgment results, and extract the unmatched original job feature information to form a bias feature set. Specifically, the following steps are included:

[0073] 2.1. Constructing feature relationship matrix Each original post

[0074] The bit information corresponds to a feature relationship matrix; among them, Represent the eigenvalues ​​of the highest age, gender, and ethnicity information, respectively. They represent the eigenvalues ​​of the minimum years of work, the minimum skill level, and the highest education level, respectively. They represent the modulus of the work content feature vector, the eigenvalue of the minimum salary feature, and the modulus of the job responsibility feature vector respectively.

[0075] Among them, the characteristic values ​​of the highest age, gender, and ethnic information are the numerical value of the highest age, the quantitative value of gender, and the unique hot coding of ethnic information; among them, the characteristic value of the minimum working years characteristic, the characteristic value of the minimum skill level characteristic, the characteristic value of the minimum salary characteristic, and the characteristic value of the highest education characteristic are the numerical value of the minimum working years, the numerical value of the minimum skill level, the numerical value of the minimum salary, and the unique hot coding of the highest education characteristic;

[0076] Among them, the work content feature vector can be expressed as ,in They represent the elements of the work content feature vector respectively; The dimension of the feature vector representing the work content; Indicates A job content feature vector;

[0077] The job responsibility feature vector can be expressed as ,in They represent the elements of the job responsibility feature vector respectively; Represents the dimension of the feature vector of job responsibilities; Indicates Position responsibility feature vector;

[0078] 2.2. Extract the row vector of the feature relationship matrix as the input vector, and after preprocessing, input it into the pre-trained random forest model to determine whether the demand characteristics match the corresponding basic characteristics and capability characteristics, and output the judgment result.

[0079] 2.3. Obtain the judgment result. If the required features do not match the corresponding basic features and capability features, find the unmatched feature pairs. or ,in Respectively represent the feature value of any basic feature in the basic feature set, capability feature set and requirement feature set;

[0080] 2.3.1. The row vector in the feature relationship matrix is ​​expressed as:

[0081] ,but The feature pairs included in are: ;in, , represents the number of rows of the feature relationship matrix, ;

[0082] 2.3.2 Calculate distance ;

[0083] if Greater than the preset distance threshold , then traverse each feature pair and calculate the distance after removing the corresponding feature pair ,if Less than , then the corresponding feature pair is judged to lead to If the distance is greater than a preset distance threshold, it means that the demand feature does not match the corresponding basic feature and ability feature; the corresponding feature pair is a biased feature pair; for example, in job information, sometimes the work content does not meet the educational requirements, that is, simple work content requires a very high level of education. In this case, it can be considered that there is a bias towards educational qualifications in the job information. At this time, the two features of work content and highest educational qualifications constitute a biased feature pair.

[0084] 2.3.3. Obtain the original job feature information corresponding to the corresponding feature pairs to form a bias feature set.

[0085] 3. Eliminate unmatched original job features from the original job feature set to form a missing job feature set; based on the missing job feature set, search for new job features that match the elements in the missing job feature set from the database or network.

[0086] The steps of excluding unmatched original job features from the original job feature set to form a missing job feature set include:

[0087] Obtaining demand characteristics and / or basic characteristics, and capability characteristics corresponding to the bias characteristics;

[0088] Remove the demand characteristics and / or basic characteristics, and ability characteristics corresponding to the biased characteristics from the original job characteristic set to form a missing job set;

[0089] Construct the feature relationship matrix corresponding to the missing job set ;

[0090] The eigenvalue or mode corresponding to the bias feature pair is The values ​​of the corresponding positions in are set to 0; for example, the bias pair is and , then the corresponding eigenvalue is and ; The characteristic relationship matrix at this time ;

[0091] Wherein, based on the missing job feature set, searching for new job features matching the elements in the missing job feature set from a database or a network includes the following steps:

[0092] Get feature relationship matrix Based on the missing job feature set, the job information corresponding to the missing job feature set is crawled from the network; for example, in this embodiment, there is a bias in the highest educational level and work content characteristics. For example, the highest educational level required for a simple text editing job is a master's degree, but most of the text editing jobs found in the database or on the network with the same age, gender, and nationality require a bachelor's degree as the highest educational level; at this time, a new job feature with a bachelor's degree as the highest educational level can be used to replace the original job feature with a master's degree.

[0093] Use TF-IDF vectorizer to vectorize the crawled job description information and convert it into job content feature vector and job responsibility feature vector;

[0094] Use the K-means algorithm to cluster the work content feature vectors and job responsibility feature vectors, calculate the proportion of bias features in each cluster, and find the cluster with the largest proportion;

[0095] Extracting new job features corresponding to the bias feature pair from the cluster with the largest proportion, obtaining the corresponding new job feature value and / or the modulus of the new feature vector to form a new job feature pair; the modulus of the new job feature value or the new feature vector is different from the value of the element in the bias feature pair;

[0096] Modular replacement using new position characteristic values ​​and new characteristic lines The corresponding position in 0.

[0097] For example, the eigenvalues ​​of the new positions and the modulus of the new eigenline are and , then replace The corresponding position of 0 is:

[0098] .

[0099] 4. Storing the new position features in the missing position feature set to form a new position feature set, repairing the original position information according to the new position feature set, and obtaining the new position information, including the following steps:

[0100] Obtain new position characteristic values ​​and new characteristic vectors; use the new position characteristic values ​​to replace the values ​​of the position characteristics in the original position characteristic set; and use the new characteristic vectors to replace the position characteristic vectors in the original position characteristic set.

[0101] 5. Use the pre-existing job recommendation model to find the information of the recommended personnel in the database that matches the new job feature set. Based on the output results of the job recommendation model, use the fairness index to evaluate the fairness of the job recommendation model to determine whether the job recommendation model has algorithmic bias; if so, adjust the parameters of the job recommendation model and re-match the recommended personnel; otherwise, proceed to step 6.

[0102] The method of using the pre-stored job recommendation model to find the recommended personnel information matching the new job feature set in the database includes the following steps:

[0103] Get the recommended personnel information in the database and extract the personnel resume information from the personnel information;

[0104] Use TF-IDF vectorizer to vectorize the job information of the resume to obtain a job position vector, where the job position vector includes the basic characteristics, ability characteristics and demand characteristics of the job seeker;

[0105] The job vector is input into the job recommendation model, and the job recommendation model combines the job vector with The row vectors in are compared and the similarity is calculated; if the similarity is greater than the recommendation threshold, it is judged that the recommended person information matches the new position feature set, otherwise, it is judged that the recommended person information does not match the new position feature set; finally, the matching result is output.

[0106] Among them, based on the output results of the job recommendation model, the fairness index is used to evaluate the fairness of the job recommendation model to determine whether the job recommendation model has algorithmic bias; if so, the parameters of the job recommendation model are adjusted and the recommended personnel are re-matched, including the following steps:

[0107] Obtain a new job feature set and check whether there is an unbalanced distribution of gender, ethnicity, age, and education level in the new job feature set to prevent one or more features from dominating the new job feature set.

[0108] Using the original job feature set, recommending job information to the recommended person through the job recommendation model;

[0109] Using the new job feature set, recommend job information to the recommended personnel through the job recommendation model;

[0110] Use mean square error (MSE) and absolute error (MAE) as fairness indicators to calculate the prediction errors of the original job feature set and the new job feature set; specifically:

[0111] Calculate the MSE and MAS of the job prediction model under the original job feature set and the new job feature set, and compare the difference values ​​of the fairness index on the original job feature set and the new job feature set;

[0112] If the difference between MSE and MAS on the new job feature set is smaller than that on the original job feature set, the new job feature set is judged to be fairer than the original job feature set;

[0113] Otherwise, adjusting the parameters of the job recommendation model; the adjusting the parameters of the job recommendation model includes one or more of hyperparameter tuning, adjusting the linear regression coefficient, and adjusting the number of layers of the neural network;

[0114] In this embodiment, the job recommendation model is constructed based on the random forest model. Therefore, when adjusting the job recommendation model, it can be done through hyperparameter tuning.

[0115] 6. Recommend the repaired original job information to the recommended personnel and feedback the repair record to the host computer, including the following steps:

[0116] The repaired original job information, that is, the new job information corresponding to the new job information set, is pushed to the matching recommended personnel; the job characteristics corresponding to the bias feature pair are fed back to the host computer; the new job information set is fed back to the host computer for the convenience of management personnel to query.

[0117] The present invention also provides a job recommendation device based on big data, including a processor, a memory connected to the processor and a communication module, wherein the processor is connected to an external host computer through the communication module, and the device is used to execute the job recommendation method based on big data.

[0118] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the job recommendation method based on big data.

[0119] In the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. The embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part, and / or installed from a removable medium. When the computer program is executed by the central processing unit (CPU), the above functions defined in the method of the present application are executed. It should be noted that the computer-readable medium mentioned above in the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more wire segments, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, electrical wire, optical cable, RF, etc., or any suitable combination of the foregoing.

[0120] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the boxes can also occur in a different order than the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0121] Those skilled in the art should understand that the embodiments of the present invention described above and shown in the accompanying drawings are only examples and do not limit the present invention. The objectives of the present invention have been fully and effectively achieved. The functions and structural principles of the present invention have been demonstrated and explained in the embodiments. Without departing from the principles, the implementation methods of the present invention may be subject to any changes or modifications.

Claims

1. A job recommendation method based on big data, characterized in that: The method comprises: Acquire original job information, extract multiple original job features from the original job information to form an original job feature set; the original job information includes one or more of age, gender, ethnicity, education, work experience, skill level and salary; Classify the elements in the original job feature set into basic features, ability features and demand features, judge whether the demand features match the basic features and ability features, conduct bias evaluation on the original job feature set based on the judgment results, and extract the unmatched original job feature information to form a bias feature set; Excluding unmatched original job features from the original job feature set to form a missing job feature set; based on the missing job feature set, searching for new job features that match the elements in the missing job feature set from a database or a network; The new position features are stored in the missing position feature set to form a new position feature set, and the original position information is repaired according to the new position feature set to obtain the new position information; Use the pre-stored job recommendation model to find the recommended personnel information in the database that matches the new job feature set. Based on the output of the job recommendation model, use the fairness index to evaluate the fairness of the job recommendation model to determine whether the job recommendation model has algorithmic bias. If so, adjust the parameters of the job recommendation model and re-match the recommended personnel. Otherwise, proceed to the next step. The repaired original job information is recommended to the recommended personnel, and the repair record is fed back to the host computer.

2. The method for job recommendation based on big data according to claim 1, characterized in that: Extract the maximum age, gender, and ethnicity from the original job information to form the basic feature set; extract the highest education, minimum years of work, and minimum skill level features to form the ability feature set; extract the highest salary, work content feature vector, and job responsibility feature vector to form the demand feature set; The extraction of the highest salary, work content feature vector and position responsibility feature vector to form a demand feature set includes: Use TF-IDF vectorization method to convert the job content description in the original job information into a job content feature vector; Use TF-IDF vectorization method to convert the job description in the original job information into a job feature vector; The extracted highest salary, work content feature vector and position responsibility feature vector constitute the demand feature set.

3. A method for job recommendation based on big data according to claim 2, characterized in that: The determining whether the demand characteristics match the basic characteristics and the capability characteristics includes: Constructing feature relationship matrix ;in, Represent the eigenvalues ​​of the highest age, gender, and ethnicity information, respectively. They represent the eigenvalues ​​of the minimum years of work, the minimum skill level, and the highest education level, respectively. They represent the modulus of the job content feature vector, the eigenvalue of the minimum salary feature, and the modulus of the job responsibility feature vector respectively; The row vector of the feature relationship matrix is ​​extracted as the input vector, and after preprocessing, it is input into the pre-trained random forest model to determine whether the demand characteristics match the corresponding basic characteristics and capability characteristics, and output the judgment result.

4. The method for job recommendation based on big data according to claim 3, characterized in that: The bias evaluation is performed on the original job feature set according to the judgment result, and the unmatched original job feature information is extracted to form a bias feature set, including: Get the judgment result. If the required features do not match the corresponding basic features and capability features, find the unmatched feature pairs. or ,in Respectively represent the feature value of any basic feature in the basic feature set, capability feature set and requirement feature set; The method of finding unmatched feature pairs comprises the following steps: The row vector in the feature relationship matrix is ​​represented as: ,but The feature pairs included in are: ;in, , represents the number of rows of the feature relationship matrix, ; Calculating distance ; if Greater than the preset distance threshold , then traverse each feature pair and calculate the distance after removing the corresponding feature pair ,if Less than , then the corresponding feature pair is judged to lead to If the distance is greater than a preset threshold, the demand feature does not match the corresponding basic feature and capability feature; the corresponding feature pair is a biased feature pair; The original job feature information corresponding to the corresponding feature pairs is obtained to form a bias feature set.

5. The method for job recommendation based on big data according to claim 4, characterized in that: The process of excluding unmatched original job features from the original job feature set to form a missing job feature set includes: Obtaining demand characteristics and / or basic characteristics, and capability characteristics corresponding to the bias characteristics; Remove the demand characteristics and / or basic characteristics, and ability characteristics corresponding to the biased characteristics from the original job characteristic set to form a missing job set; Construct the feature relationship matrix corresponding to the missing job set ; The eigenvalue or modulus corresponding to the biased feature pair is The value of the corresponding position in is set to 0.

6. A method for job recommendation based on big data according to claim 5, characterized in that: The method of searching for new job features matching the elements in the missing job feature set from a database or a network based on the missing job feature set includes the following steps: Get feature relationship matrix , Based on the missing job feature set, crawl the job information corresponding to the missing job feature set from the network; Use TF-IDF vectorizer to vectorize the crawled job description information and convert it into job content feature vector and job responsibility feature vector; Use the K-means algorithm to cluster the work content feature vectors and job responsibility feature vectors, calculate the proportion of bias features in each cluster, and find the cluster with the largest proportion; Extracting new job features corresponding to the bias feature pair from the cluster with the largest proportion, obtaining the corresponding new job feature value and / or the modulus of the new feature vector to form a new job feature pair; the modulus of the new job feature value or the new feature vector is different from the value of the element in the bias feature pair; Modular replacement using new position characteristic values ​​and new characteristic lines The 0 value at the corresponding position in ; The storing of the new position features into the missing position feature set to form a new position feature set, and repairing the original position information according to the new position feature set, includes: Obtain new position feature values ​​and new feature vectors; Use the new position characteristic values ​​to replace the values ​​of the position characteristics in the original position characteristic set; Use the new feature vector to replace the position feature vector in the original position feature set.

7. The method for job recommendation based on big data according to claim 6, characterized in that: The method of using the pre-stored job recommendation model to search for the recommended personnel information in the database that matches the new job feature set includes: Get the recommended personnel information in the database and extract the personnel resume information from the personnel information; Use TF-IDF vectorizer to vectorize the job information of the resume to obtain a job position vector, where the job position vector includes the basic characteristics, ability characteristics and demand characteristics of the job seeker; The job vector is input into the job recommendation model, and the job recommendation model combines the job vector with The row vectors in are compared and the similarity is calculated; if the similarity is greater than the recommendation threshold, it is judged that the recommended person information matches the new position feature set, otherwise, it is judged that the recommended person information does not match the new position feature set; finally, the matching result is output.

8. The method for job recommendation based on big data according to claim 7, characterized in that: Based on the output results of the job recommendation model, the fairness of the job recommendation model is evaluated using fairness indicators to determine whether there is algorithmic bias in the job recommendation model; If so, adjust the parameters of the job recommendation model and re-match the recommended personnel, including: Obtain a new job feature set and check whether there is an unbalanced distribution of gender, ethnicity, age, and education level in the new job feature set to prevent one or more features from dominating the new job feature set. Using the original job feature set, recommending job information to the recommended person through the job recommendation model; Using the new job feature set, recommend job information to the recommended personnel through the job recommendation model; Use mean square error (MSE) and absolute error (MAE) as fairness indicators to calculate the prediction errors of the original job feature set and the new job feature set; specifically: Calculate the MSE and MAS of the job prediction model under the original job feature set and the new job feature set, and compare the difference values ​​of the fairness index on the original job feature set and the new job feature set; If the difference between MSE and MAS on the new job feature set is smaller than that on the original job feature set, the new job feature set is judged to be fairer than the original job feature set; Otherwise, adjusting the parameters of the job recommendation model; the adjusting the parameters of the job recommendation model includes one or more of hyperparameter tuning, adjusting the linear regression coefficient, and adjusting the number of layers of the neural network; The process of recommending the repaired original job information to the recommended personnel and feeding back the repair record to the host computer includes: Push the new job information corresponding to the new job information set to the matching recommended personnel; Feedback the job characteristics corresponding to the bias characteristics to the host computer; Feedback the new position information set to the host computer for management personnel to query.

9. A job recommendation device based on big data, comprising a processor, a memory connected to the processor and a communication module, wherein the processor is connected to an external host computer through the communication module, characterized in that: The device is used to execute a job recommendation method based on big data as described in any one of claims 1 to 8.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and the computer program is executed by a processor to implement a job recommendation method based on big data as described in any one of claims 1 to 8.

Images