Recruitment effect estimation method, electronic equipment and storage medium

Abstract

The invention discloses a recruitment effect estimation method, electronic equipment and a storage medium, and the method comprises the steps: obtaining positions having the same or similar function information with a to-be-evaluated position, and obtaining an extended position set; obtaining a sample data set of the to-be-evaluated position and the extended position set in a preset time period of the recruitment platform; executing the recommendation request on the to-be-evaluated position in a pre-constructed simulation system to obtain a sorting score of the to-be-evaluated position in a recommendation list; sorting the plurality of positions of each piece of sample data to obtain a plurality of position sorting queues; judging whether the to-be-evaluated position is exposed in each piece of sample data or not according to the sorting scores of the to-be-evaluated position in the recommendation lists of the different job seekers and the plurality of position sorting queues; and counting the exposure times of the to-be-evaluated position in the sample data set to determine the delivery quantity of the to-be-evaluated position. By simulating the recommendation request for the to-be-evaluated position in the simulation system, the delivery amount of the to-be-evaluated position is calculated to be accurate.

Description

Technical Field

[0001] The present application relates to the technical field of computers and the Internet, and in particular to a recruitment effect prediction method, electronic device, and storage medium. Background Art

[0002] When purchasing value-added recruitment services, customers are most concerned about whether the purchase of the value-added services can lead to an increase in delivery volume and what the return on investment (ROI) is. These delivery volume growth effects and ROI estimates are very important for customers' decision-making on purchasing recruitment growth services. In existing businesses, the results of similar positions are often used as a substitute for effect estimates, or estimates are made based on traffic within the grid. There is a significant difference between the estimated effects and the actual effects of these methods, which may have the following adverse effects on business and product development: 1. If the effect of the value-added service purchased by the customer is poor and does not bring the expected effect to the customer, it will have a negative impact on the overall reputation and sustainable development of the value-added service; 2. If the effect of the value-added service purchased by the customer is relatively good, but the customer does not have an accurate effect estimate and ROI data reference during the purchase decision stage, it may lead to abandonment of the purchase. Summary of the Invention

[0003] The purpose of this application is to provide a recruitment effect estimation method, electronic device and storage medium.

[0004] This embodiment of the present application provides a recruitment effect estimation method, including:

[0005] Obtain positions with the same or similar functional information as the position to be evaluated to obtain an expanded position set;

[0006] Obtaining a sample data set of the positions to be evaluated and the expanded position set within a preset time period on the recruitment platform; wherein each sample data in the sample data set corresponds to a different job seeker, and each sample data includes multiple position data browsed by the same job seeker;

[0007] Executing a recommendation request for the position to be evaluated in a pre-built simulation system to obtain a ranking score of the position to be evaluated in a recommendation list; wherein the simulation system includes a user context simulation module and an online environment simulation module; the user context simulation module is used to extract context feature data of each job seeker visiting the recruitment platform from the recruitment platform's log system; the online environment simulation module is used to execute a recommendation request for the position to be evaluated based on the context feature data corresponding to each job seeker, to obtain a ranking score of the position to be evaluated in the recommendation list of different job seekers;

[0008] Sort the multiple positions of each sample data in the sample data set according to the scores of the first sorting model to obtain multiple position sorting queues; wherein each position sorting queue corresponds to each sample data one by one;

[0009] According to the ranking scores of the positions to be evaluated in the recommendation lists of different job seekers, the positions to be evaluated are placed in a plurality of the position ranking queues to determine whether the positions to be evaluated can be exposed in each sample data;

[0010] Count the number of times the position to be evaluated is exposed in the sample data set to determine the number of applications for the position to be evaluated.

[0011] Furthermore, the online environment simulation module includes a first recall model and a second ranking model; executing a recommendation request for the position to be evaluated in the pre-built simulation system to obtain a ranking score of the position to be evaluated in the recommendation list includes:

[0012] Input the context feature data of the current sample data into the online environment simulation module, and use the first recall model to perform recall scoring on the position to be evaluated to obtain a first recall score;

[0013] Obtaining a job recall queue of current sample data in a log system of the recruitment platform; wherein the job recall queue of the current sample data is a sorted queue obtained by respectively performing recall scoring on multiple jobs browsed by the current job seeker according to a second recall model of the recruitment platform, and sorting the jobs according to the recall scores;

[0014] Based on the first recall score and the job recall queue of the current sample data, the matching degree between the current job seeker and the job to be evaluated is determined;

[0015] If the matching degree between the current job seeker and the position to be evaluated meets the preset conditions, the current sample data will be retained;

[0016] Executing a recommendation request for the position to be evaluated in the context feature data of the retained current sample data to generate a recommendation list for the current job seeker;

[0017] The recommendation list of the current job seeker is sorted and scored using the second sorting model to obtain a sorting score of the position to be evaluated in the recommendation list of the current job seeker.

[0018] Furthermore, the process of placing the positions to be evaluated in a plurality of position ranking queues according to the ranking scores of the positions to be evaluated in the recommendation lists of different job seekers, so as to determine whether the positions to be evaluated can be exposed in each sample data, includes:

[0019] Determine the target ranking position of the position to be evaluated in the position ranking queue of the current sample data based on the ranking score of the position to be evaluated in the recommendation list of the current job seeker and the position ranking queue of the current sample data;

[0020] If the target ranking position is within the preset browsing depth of the current sample data, the position to be evaluated is determined to be exposed in the current sample data;

[0021] Traverse each sample data in the sample data set to obtain the total number of times the position to be evaluated is exposed in the sample data set.

[0022] Furthermore, the context feature data of the job seeker when visiting the recruitment platform includes one or more of the following: resume ID, job position, time, user portrait data, user behavior scenario data and page entry source parameters.

[0023] Furthermore, the method of counting the number of exposures of the position to be evaluated in the sample data set to determine the number of applications for the position to be evaluated includes:

[0024] When it is determined that the position to be evaluated is exposed in the current sample data, the current estimated viewing rate and the current estimated conversion rate of the position to be evaluated output by the second ranking model are obtained;

[0025] The current delivery rate is calculated based on the current estimated view rate and the current estimated view-to-delivery conversion rate;

[0026] Traverse each sample data exposed in the sample data set to obtain the total delivery rate;

[0027] The delivery volume for the position to be evaluated is calculated based on the total delivery rate and the sum of the exposure times.

[0028] Furthermore, the method further comprises:

[0029] Screen out high-potential positions from multiple candidate positions and use them as positions to be evaluated; among them,

[0030] High-potential positions include positions with a staffing ratio below a set threshold, positions that are in urgent need of recruitment, or positions with a high probability of purchasing value-added services predicted by the model.

[0031] Furthermore, the step of obtaining positions having the same or similar functional information as the position to be evaluated to obtain an expanded position set includes:

[0032] Obtaining the functional information of the position to be evaluated, and expanding the functional information of the position to be evaluated to obtain similar functional information;

[0033] Carry out multi-level functional classification of the positions to be evaluated to obtain the target functional positions;

[0034] Combine the target functional position with the regional information to obtain a target functional position grid; wherein the functional information of the position to be evaluated includes regional information;

[0035] Through the graph embedding algorithm, the target functional position grid is expanded using the similarity grid to obtain an extended position set; wherein the similarity grid is the grid where positions with the same or similar functional information as the position to be evaluated are located.

[0036] Furthermore, the step of obtaining a sample data set of the expanded position grid set within a preset time period of the recruitment platform includes:

[0037] If the number of samples corresponding to the currently expanded position grid obtained is greater than the set threshold, sample data is randomly selected to reduce the number of samples.

[0038] An embodiment of the present application provides an electronic device, which includes a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the steps of the method described above are implemented.

[0039] An embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the steps of the method described above are implemented.

[0040] An embodiment of the present application provides a computer program product, which includes computer program instructions. When the computer program instructions are executed by a processor, the steps of the method described above are implemented.

[0041] The above technical solution of this application has the following beneficial technical effects:

[0042] In the embodiment of the present application, by simulating user browsing behavior and taking into account the possibility of job seekers submitting similar positions, the exposure order on the real information flow and the browsing depth of job seekers, etc., the calculated submission volume of the position to be evaluated is accurate, and the calculated submission volume of the position to be evaluated is compared with the submission volume of the position on the recruitment platform, an accurate estimate of the recruitment effect of the increase in submission volume can be obtained, and the return on investment (ROI) can be calculated based on the cost invested in purchasing advertising promotion services and the submission volume of the position to be evaluated. In this way, the recruitment effect of the increase in submission volume and the estimate of ROI can be used as a reference for customers when making decisions on purchasing value-added recruitment services, so as to help customers make reasonable decisions on purchasing value-added recruitment services. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings in the embodiments of the present application.

[0044] Figure 1 It is a schematic diagram of the system architecture of an embodiment of the present application.

[0045] Figure 2 This is a flowchart of a recruitment effect estimation method according to an embodiment of the present application.

[0046] Figure 3 This is a schematic diagram of the processing process of another recruitment effect estimation method in an embodiment of the present application.

[0047] Figure 4 This is a schematic diagram of an electronic device used to implement a recruitment effect estimation method according to an embodiment of the present application. DETAILED DESCRIPTION

[0048] The principles and spirit of the present application will be described below with reference to several exemplary embodiments. It should be understood that the purpose of providing these embodiments is to make the principles and spirit of the present application clearer and more thorough, so that those skilled in the art can better understand and implement the principles and spirit of the present application. The exemplary embodiments provided herein are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments herein, all other embodiments obtained by those of ordinary skill in the art without creative work are within the scope of protection of this application.

[0049] The embodiments of the present application relate to terminal devices and / or servers. Those skilled in the art will appreciate that the embodiments of the present application may be implemented as a system, apparatus, device, method, computer-readable storage medium, or computer program product. Therefore, the present disclosure may be specifically implemented in at least one of the following forms: complete hardware, complete software, or a combination of hardware and software. According to the embodiments of the present application, the present application seeks protection for a recruitment effect estimation method, apparatus, electronic device, computer-readable storage medium, and computer program product. Figure 1 A schematic diagram of a system architecture of an embodiment of the present application is shown. Figure 1As shown, the system includes a terminal device 102 and a server 104. The terminal device 102 may include at least one of the following: a smartphone, a tablet computer, a laptop computer, a desktop computer, a smart TV, various wearable devices, an augmented reality (AR) device, a virtual reality (VR) device, and the like. A client may be installed on the terminal device 102. For example, the client may be a client dedicated to performing a specific function (such as an application app), or a client embedded with multiple application applets (with different functions), or a client logged in through a browser. The user may perform operations on the terminal device 102. For example, the user may open the client installed on the terminal device 102 and input instructions through the client operation, or the user may open the browser installed on the terminal device 102 and input instructions through the browser operation. After receiving the instruction input by the user, the terminal device 102 sends a request message containing the instruction to the server 104. After receiving the request message, the server 104 performs the corresponding processing and then returns the processing result information to the terminal device 102. The user instruction is completed through a series of data processing and information exchange.

[0050] In this document, terms such as first, second, and third are only used to distinguish one entity (or operation) from another entity (or operation), and are not intended to require or imply any order or relationship between these entities (or operations).

[0051] When purchasing value-added recruitment services, customers are most concerned about whether the services can increase the number of applications delivered and what the return on investment (ROI) ratio is. These delivery volume growth effects and ROI estimates are crucial to customers' decision-making regarding recruitment growth services. Existing technical solutions for estimating delivery volume growth and ROI are as follows:

[0052] Solution 1: Select a position similar to the position to be evaluated that has purchased value-added services, and use the recruitment performance and ROI of the similar position as the performance or ROI of the position to be evaluated. The disadvantages of this solution are: 1) the recruitment results are inaccurate; 2) if there is no similar position that has purchased value-added services, this solution cannot be used.

[0053] Solution 2: Calculate the number of job seekers for the position to be evaluated. Based on the application rate, calculate the resume data that can be obtained for the position, and calculate the recruitment effect and ROI. The disadvantage of this solution is that the recruitment effect is extremely inaccurate. The main reasons for this disadvantage are as follows: 1) It does not consider the application volume of similar positions. For example, candidates applying for similar positions may also apply for the position; 2) It does not consider the influence of factors such as the order of job exposure in the real information flow and the depth of user browsing.

[0054] Based on this, an embodiment of the present application proposes a recruitment effect estimation method to accurately estimate the recruitment effect after the user purchases the value-added service.

[0055] Figure 2 A flowchart of a recruitment effect estimation method according to an embodiment of the present application is shown, and the method includes the following specific steps:

[0056] S120: Obtain positions that have the same or similar functional information as the position to be evaluated to obtain an extended position set.

[0057] Specifically, the position to be evaluated can be one that has already been posted on a recruitment platform. When a job is posted on a recruitment platform app by a recruiter without purchasing value-added recruitment services (i.e., advertising and promotion services), unsatisfactory recruitment results may result. The functional information of the position to be evaluated can include, for example, the following elements: position title, department, job responsibilities, job requirements, work location, salary range, etc. This information can be expanded based on each element, with functional requirements that have a certain degree of similarity or a high degree of match to each element being considered as similar functional information for the position to be evaluated.

[0058] For example, if Company A posts a "Financial Accountant" position on the recruitment platform app, different recruitment entities such as Company B and Company C can post "Financial Accountant" positions on the recruitment platform app for recruitment. Therefore, the "Financial Accountant" positions posted by different recruitment entities such as Company B and Company C can be used as extended positions; and recruitment positions posted by other recruitment entities, such as Financial Analyst, General Ledger Manager and Financial Advisor, etc. These recruitment positions are the same or similar to "Financial Accountant" in terms of job responsibilities, job requirements, work location, salary range and other factors, that is, they have the same or similar functional information as the "Financial Accountant" position of Company A. Therefore, the above recruitment positions can all be used as extended positions of the "Financial Accountant" of Company A; among which, the extended position is a position with the same or similar functional information as the position to be evaluated.

[0059] S130: Obtain a sample data set of the positions to be evaluated and the extended position set within a preset time period of the recruitment platform; wherein each sample data in the sample data set corresponds to a different job seeker, and each sample data includes multiple position data browsed by the same job seeker.

[0060] Specifically, the preset time period is the time period closest to the current time, such as sample data from yesterday, or sample data from the previous few days. The log system of the recruitment platform can store daily user behavior log data. User behavior log data may include basic access information, browsing and navigation data, search data, and job interaction data, etc. Job interaction data may include information such as job exposure, browsing, clicking, collection, and resume submission. It is possible to directly obtain from the log system of the recruitment platform which job seekers browsed the positions to be evaluated and the extended position set within the preset time period, and then classify the multiple positions browsed by the job seekers according to the user ID (or resume ID) of the job seekers, so as to obtain multiple position data browsed by the same job seeker. For example, the positions browsed by job seeker Zhang San in sample data 1 may include Shanghai accountant of company A, Shanghai financial analyst of company B, Shanghai general ledger supervisor of company C, Shanghai accountant of company D, etc.

[0061] S140: Executing a recommendation request for the position to be evaluated in a pre-built simulation system to obtain a ranking score of the position to be evaluated in a recommendation list; wherein, the simulation system includes a user context simulation module and an online environment simulation module; the user context simulation module is used to extract context feature data of each sample data when the job seeker visits the recruitment platform from the log system of the recruitment platform; the online environment simulation module is used to execute a recommendation request for the position to be evaluated based on the respective context feature data corresponding to different job seekers to obtain a ranking score of the position to be evaluated in the recommendation list of different job seekers.

[0062] Specifically, the contextual feature data of a job seeker's browsing experience during a visit to a recruitment platform may include, for example, the user's profile, job search expectations, browsing scenario, server-retrieved positions, a ranked job list, a front-end exposed job list, the sequence of positions viewed by the user, clicked positions, and submitted positions. By acquiring the contextual feature data of real job seekers visiting the recruitment platform app and inputting this contextual feature data into an online environment simulation module, a simulation of the position to be evaluated purchasing advertising promotion services, i.e., executing a recommendation request for the position to be evaluated, can be performed. The ranking of the position to be evaluated in the recommendation lists of different job seekers can be obtained, and a ranking score for the position to be evaluated can be calculated using a ranking model. The ranking model incorporates the real-time browsing scenario features of the job seeker, such as whether the job seeker is currently on the search page or the recommendation page, to ensure that the generated recommendation list for the job seeker conforms to actual user behavior patterns.

[0063] S150: Sort the multiple positions of each sample data in the sample data set according to the scoring values of the first sorting model to obtain multiple position sorting queues; wherein each position sorting queue corresponds to each sample data one by one.

[0064] Specifically, the job ranking queue for the current sample data can be obtained from the recruitment platform's log system. This ranking queue is obtained by fine-scoring the multiple jobs viewed by the current job seeker using the recruitment platform's ranking model, and then sorting them according to the scores. For example, the ranking model can be a deep neural network model such as DeepFM, Wide&Deep, or MMoE. Multiple job ranking queues are the ranking queues of jobs viewed by different job seekers in the recruitment platform's app.

[0065] S160: placing the positions to be evaluated in a plurality of position ranking queues according to the ranking scores of the positions to be evaluated in the recommendation lists of different job seekers, so as to determine whether the positions to be evaluated can be exposed in each sample data.

[0066] Specifically, by comparing the ranking score of the position to be evaluated in the current job seeker's recommendation list with the ranking scores of each position in the current job seeker's position ranking queue, the position to be evaluated is inserted into the current job seeker's position ranking queue, and the target ranking position of the position to be evaluated in the current job seeker's position ranking queue can be determined; wherein, the ranking scores of each position in the position ranking queue are arranged in descending order; if the target ranking position is within the current job seeker's preset browsing depth, the position to be evaluated can be displayed to the current job seeker, that is, the position to be evaluated can be exposed in the sample data; this can simulate the competitive environment of job seekers in the real recommendation list, avoid the defect of traditional methods that ignore the influence of ranking position, and thus accurately simulate the real information flow competition environment. wherein, the preset browsing depth is the number of positions browsed by the job seeker in a search request or recommendation request, and the preset browsing depth is less than or equal to the number of positions in the corresponding position ranking queue.

[0067] S170: Count the number of times the position to be evaluated is exposed in the sample data set to determine the delivery volume of the position to be evaluated.

[0068] In the embodiment of the present application, similarity expansion is performed on the functional information of the position to be evaluated to obtain an extended position set of the position to be evaluated, and then a real sample data set of the position to be evaluated and the extended position within a preset time period of the recruitment platform is obtained, and the context feature data of the job seekers browsing the positions when visiting the recruitment platform is input into the online environment simulation module, and after simulating the purchase of advertising promotion services for the position to be evaluated, a recommendation request is executed for the position to be evaluated, and the ranking of the position to be evaluated in the recommendation lists of different job seekers can be obtained and the ranking score of the position to be evaluated can be calculated through the ranking model, and the position to be evaluated can be inserted into the job seeker's position ranking queue according to the ranking score of the position to be evaluated; the ranking position of the position to be evaluated in the job seeker's position ranking queue and the job seeker's actual browsing depth can be combined to determine whether the position to be evaluated is exposed, and the position to be evaluated can be statistically analyzed. The number of exposures in the sample data set is calculated to obtain the total number of exposures, and the delivery volume of the position to be evaluated is calculated in combination with the total delivery rate of the position to be evaluated; therefore, by simulating user browsing behavior and taking into account the possibility of job seekers delivering similar positions, the exposure order on the real information flow and the browsing depth of job seekers, the calculated delivery volume of the position to be evaluated is accurate, and the calculated delivery volume of the position to be evaluated is compared with the delivery volume of the position on the recruitment platform, an accurate estimate of the recruitment effect of the increase in delivery volume can be obtained, and the return on investment (ROI) can be calculated based on the cost of purchasing advertising promotion services and the delivery volume of the position to be evaluated. In this way, the recruitment effect of the increase in delivery volume and the estimate of ROI can be used as a reference for customers when making decisions on purchasing recruitment value-added services, so as to help customers make reasonable decisions on purchasing recruitment value-added services.

[0069] In some embodiments, step S120: obtaining positions having the same or similar functional information as the position to be evaluated to obtain an extended position set includes the following specific steps:

[0070] S121: Acquire the functional information of the position to be evaluated, and expand the functional information of the position to be evaluated to obtain similar functional information;

[0071] S122: Perform multi-level functional classification on the position to be evaluated to obtain the target functional position;

[0072] S123: combining the target functional position and the regional information to obtain a target functional position grid; wherein the functional information of the position to be evaluated includes regional information;

[0073] S124: By using a graph embedding algorithm, the target functional position grid is expanded using a similarity grid to obtain an expanded position set; wherein the similarity grid is a grid containing positions having the same or similar functional information as the position to be evaluated.

[0074] Specifically, obtaining the functional information of the position to be evaluated may include, for example, the following elements: position title, department, job responsibilities, job requirements, work location, salary range, etc., among which the work city can be used as geographical information, which can be expanded according to each element, and functional requirements that have a certain similarity with each element, or a high degree of matching, are used as similar functional information of the position to be evaluated. Functional category (three-level classification) + work city form a grid, which can be used as the minimum precise matching unit for matching job seekers and positions. For example, a recruiter posts a job position for "Financial Accountant" on the recruitment platform app. The functional information of this position includes the work city, etc. For example, the work city of the "Financial Accountant" position is "Shanghai". The "Financial Accountant" is functionally classified into "Finance / Finance / Accountant" according to the three-level category, and the subdivided functional position is "Accountant". After combining the functional position with the geographical information, the target functional position grid "Shanghai Accountant" can be obtained. Expand the grid containing "Shanghai Accountant" and use the similarity grid to expand the target functional position grid. First, determine the functional information and grid containing "Shanghai Accountant." Then, based on a graph embedding algorithm (GraphEmbedding), generate similar functions and similar grids. For example, using historical user behavior data, a model is trained using a Graph Embedding network. Once trained, this model can generate a similarity grid, i.e., a grid containing positions with the same or similar functional information as the position to be evaluated. This yields a set of expanded position grids for "Shanghai Accountant." By expanding the target functional position grid, more positions similar or close to the position to be evaluated can be obtained. Sample data for these expanded positions can then be obtained. This not only simulates the real-world online recruitment platform, but also effectively reduces the system's computational load compared to obtaining all sample data from the recruitment platform while ensuring recruitment effectiveness.

[0075] In some embodiments, step S130: obtaining a sample data set of the expanded job set within a preset time period on the recruitment platform includes the following specific steps:

[0076] If the number of samples corresponding to the current expanded position is greater than the set threshold, sample data is randomly selected to reduce the number of samples.

[0077] Specifically, if the sample size for the currently expanded position "Shanghai Financial Analyst" is large, meaning many different companies are hiring for this position and the number of resumes submitted by job seekers is increasing (i.e., exceeding a set threshold), a certain number of submitted resumes will be randomly sampled as sample data to reduce the sample size and the system's computational load. Because these job openings target similar job seekers with similar historical user behavior data, the sampling ratio can be used to recover the true recruitment results.

[0078] In some embodiments, the online environment simulation module includes a first recall model and a second ranking model. Step S140: executing a recommendation request for the position to be evaluated in the pre-built simulation system to obtain a ranking score of the position to be evaluated in the recommendation list includes the following specific steps:

[0079] S141: Inputting the context feature data of the current sample data into the online environment simulation module, and performing a recall score on the position to be evaluated using the first recall model to obtain a first recall score;

[0080] S142: Obtaining a job recall queue for current sample data from a log system of the recruitment platform; wherein the job recall queue for current sample data is a sorted queue obtained by performing recall scoring on multiple jobs browsed by the current job seeker according to a second recall model of the recruitment platform, and sorting the jobs according to the recall scores;

[0081] S143: Determine the matching degree between the current job seeker and the position to be evaluated based on the first recall score and the position recall queue of the current sample data;

[0082] S144: If the matching degree between the current job seeker and the position to be evaluated meets the preset conditions, the current sample data is determined to be retained;

[0083] S145: Executing a recommendation request for the position to be evaluated in the context feature data of the retained current sample data, and generating a recommendation list for the current job seeker;

[0084] S146: Sorting and scoring the recommendation list of the current job seeker using the second ranking model to obtain a ranking score of the position to be evaluated in the recommendation list of the current job seeker.

[0085] Specifically, job recall technology refers to the process of selecting job information that matches user needs and preferences from a large number of job databases. The recall model of the simulation system can adopt the DSSM recall model, input the contextual feature data of the current sample data into the online environment simulation module, request the recall model, and use the recall model to recall and score the job to be evaluated; the recall model of the recruitment platform can also adopt the DSSM recall model, and load the recall model of the recruitment platform with the corresponding version number of the sample data date into the simulation system, so as to simulate the real online recall situation of the recruitment platform; when the job to be evaluated is published in the simulation system, and an advertising promotion service is published for the job to be evaluated, that is, when a recommendation request is executed for the job to be evaluated, the recall model of the simulation system is used to recall and score the job to be evaluated, and The recall scores of the positions in the current position recall queue are compared with the recall scores of the positions in the current position recall queue to determine the ranking position of the position to be evaluated in the current position recall queue, and based on this, it can be judged whether the current job seeker matches the position to be evaluated. The recall scores of the positions in the position recall queue are sorted from high to low. If the position to be evaluated is ranked low in the current position recall queue and does not meet the preset matching conditions, the current sample data is discarded. If the position to be evaluated is ranked high in the current position recall queue and meets the preset matching conditions, the current sample data is retained and the subsequent ranking of the positions to be evaluated in the current recommendation list is performed. The ranking model of the recruitment platform with the corresponding version number of the sample data date can be loaded into the simulation system, so that the real online ranking of the recruitment platform can be simulated.

[0086] In some embodiments, step S160: placing the position to be evaluated in a plurality of position ranking queues according to the ranking scores of the position to be evaluated in the recommendation lists of different job seekers to determine whether the position to be evaluated can be exposed in each sample data includes the following specific steps:

[0087] S161: Determine a target ranking position of the position to be evaluated in the position ranking queue of the current sample data based on the ranking score of the position to be evaluated in the recommendation list of the current job seeker and the position ranking queue of the current sample data;

[0088] S162: If the target ranking position is within the preset browsing depth of the current sample data, then the position to be evaluated is determined to be exposed in the current sample data;

[0089] S163: Traverse each sample data in the sample data set to obtain the total number of times the position to be evaluated is exposed in the sample data set.

[0090] Specifically, for example, the preset browsing depth of the current sample data is 50 positions browsed in a search request or recommendation request of the current job seeker. If the target sorting position is the 65th in the position sorting queue of the current sample data, it is determined that the position to be evaluated is not exposed in the current sample data; if the target sorting position is the 35th in the position sorting queue of the current sample data, it is determined that the position to be evaluated is exposed in the current sample data; repeat the above steps S140-S170, traverse each sample data in the sample data set, to determine whether the position to be evaluated is exposed in the current sample data; the number of exposures of the position to be evaluated in the sample data set can be counted to obtain the total number of exposures.

[0091] In some embodiments, step S170: counting the number of exposures of the position to be evaluated in the sample data set to determine the number of applications for the position to be evaluated, includes the following specific steps:

[0092] S171: When it is determined that the position to be evaluated is exposed in the current sample data, the current estimated viewing rate and the current estimated conversion rate of the position to be evaluated output by the second ranking model are obtained;

[0093] S172: Calculate the current delivery rate based on the current estimated viewing rate and the current estimated conversion rate from viewing to delivery;

[0094] S173: Traverse each exposed sample data in the sample data set to obtain a total delivery rate;

[0095] S174: The delivery volume for the position to be evaluated is calculated based on the total delivery rate and the sum of the exposure times.

[0096] Specifically, when it is determined that the position to be evaluated is exposed in the current sample data, the current estimated viewing rate and the current estimated conversion rate of views to submissions for the position to be evaluated can be output according to the ranking model in the simulation system; the delivery rate of each exposed sample data is summed to obtain the total delivery rate; the total delivery rate is multiplied by the total number of exposures to calculate the delivery volume of the position to be evaluated after purchasing the advertising promotion service. Among them, the second ranking model is the ranking model in the simulation system, for example, a trained MMOE model. The MMOE model is a deep DNN model (deep neural network model). User feature data (including resume information, job seeker profile, job seeker's historical behavior data on 51app, etc.), position feature data (including position information, position profile, etc.), and context information (including job seeker browsing time, pages, etc.) are input to train the deep DNN model. The trained MMOE model can generate scores for the current estimated viewing rate (pctr) and the current estimated conversion rate (pcvr) of views to submissions.

[0097] In some embodiments, the method may further include the following specific steps:

[0098] S110: Screen high-potential positions from multiple candidate positions and designate them as positions to be evaluated. These high-potential positions include positions with a job-to-job ratio below a set threshold, positions in urgent need of recruitment, or positions predicted by the model to have a high probability of purchasing value-added services. If a position is posted on the recruitment platform app by a recruiter without purchasing value-added services (i.e., advertising and promotion services), this may result in unsatisfactory recruitment results. Recruiters can then inquire about the recruitment results of the position after purchasing value-added services, and the position can be treated as a position to be evaluated. Furthermore, high-conversion positions can be identified and the technical solution of this application can be used to estimate the recruitment results that these positions can achieve after purchasing value-added services. This can be used for targeted investment promotion and expand the business development of the recruitment platform. The job-to-job ratio is the ratio of the number of job seekers to the number of positions within a grid. Positions within a grid with a low job-to-job ratio have fewer candidates, more positions, and fierce competition among companies for recruitment. Therefore, companies are more likely to purchase value-added products to complete recruitment tasks. Positions with a more urgent need for recruitment are also more likely to purchase value-added services.

[0099] In some embodiments, the context feature data of the job seeker when visiting the recruitment platform includes one or more of the following: resume ID, job position, time, user portrait data, user behavior scenario data, and page entry source parameters.

[0100] User portrait data may specifically include: basic information: age, gender, educational background, years of work experience, skill tags, current position, etc.; dynamic information: historical click / delivery records, search keywords, recent active time periods, preferred industries / job types, etc.; job-seeking intentions: expected salary, target city, job-seeking urgency (such as the "urgent job search" label), etc.

[0101] User behavior scenario data, that is, the context of the user's current browsing environment, may specifically include: device information: device type (mobile phone / PC), operating system, screen resolution, network status (Wi-Fi / 4G), etc.; time scenario data: browsing time (weekdays / weekends, daytime / nighttime), length of stay, etc.; interactive behavior data: page sliding speed, length of stay on the job card, whether to expand the job details, etc.

[0102] Page entry source parameters may include search pages, recommended pages, push notifications, etc.

[0103] The above describes the implementation of the present application and the advantages brought by the embodiments through multiple embodiments. Figure 3 , describes in detail the specific processing process of the embodiment of this application.

[0104] S21. Playback sample selection

[0105] Selecting a playback sample may include the following specific steps:

[0106] S211. Identify positions to be evaluated

[0107] High-potential positions can be screened from the published job positions on the recruitment platform as positions to be evaluated. For example, some high-conversion positions can be identified as positions to be evaluated. The technical solution of this application can be used to estimate the recruitment effect that can be achieved after purchasing value-added services for these positions, and then used for targeted investment promotion. If the positions to be evaluated have been determined, such as the company HR directly asking about the recruitment effect of a specific position after purchasing value-added services, the specific position can be used as the position to be evaluated, and this step can be skipped. The following two methods can be used to screen high-potential positions:

[0108] 1. Use statistical methods: Screening is done through rules based on data analysis and operational sales experience. Specific rules include positions in the grid with a low man-to-job ratio (the ratio of the number of job candidates to the number of positions in the grid) (few candidates, many positions, fierce competition in recruitment, and a greater tendency to purchase value-added products to complete recruitment tasks), and positions that are more urgently recruited (HR responds to applications for this position in a short time, the proportion of chats about this position is high, and the number of times HR proactively reaches out, etc.).

[0109] 2. How to use the model: Use a tree model to estimate the conversion rate of job recruitment. For example, based on the historical data of job positions that purchased value-added products, a model is trained. After the model is trained, it can output the probability of a specific job position purchasing value-added products.

[0110] For example, determine the title of the position to be evaluated: Financial Accounting;

[0111] Functional positions to be assessed (Level 3 category): Accountant (Finance / Accountant);

[0112] The working city of the position to be evaluated: Shanghai.

[0113] Therefore, the target functional position grid for the position to be evaluated can be determined as "Shanghai Accountant".

[0114] S212. Similar grid screening

[0115] The real user historical behavior data in the app of the simulated recruitment platform, which may specifically include data such as browsing positions, job exposure, and submitting resumes, can then be used to screen out job seekers who have the potential to apply for the positions to be evaluated.

[0116] First, the target position's functional information and grids can be determined. Similar functions and grids can then be generated using Graph Embedding. For example, a model can be trained using a Graph Embedding network using historical user behavior data. This trained model can generate functional similarity and grid similarity, resulting in an expanded set of position grids for the position being evaluated.

[0117] For example, using Graph Embedding to generate grid similarity, the "Shanghai Accountant" grid can be expanded to include Shanghai Financial Analyst, Shanghai General Ledger Manager, Shanghai Financial Advisor, Shanghai Cost Manager, Shanghai Financial Specialist, Shanghai CPA, Shanghai Audit Manager / Supervisor, Shanghai Tax Specialist, Shanghai Audit Specialist, etc. Grid expansion can simulate real-world online recall and exposure logic.

[0118] S213. Playback sample selection

[0119] Simulate the real user historical behavior data in the recruitment platform app to screen out job seekers who have potential to apply for the position to be evaluated.

[0120] After determining the grids and extended grids where the target functional position is located, filter the sample data of a certain day within these grids. Based on specific needs, you can select samples from multiple days or data from a certain day. This embodiment takes yesterday as an example, and the following content also takes yesterday as an example.

[0121] Random sampling is performed on grids with a relatively large number of samples. For example, if the number of samples is greater than 5,000, the number of samples is randomly sampled to 5,000. For example, if the number of samples corresponding to the current extended position "Shanghai Financial Analyst" is large, that is, many different corporate entities are recruiting for the position of "Shanghai Financial Analyst", and the number of resumes submitted by job seekers increases accordingly, that is, when it is greater than the set threshold (5,000), 5,000 resumes are randomly sampled as sample data for processing, in order to reduce the number of samples and reduce the amount of system computing. Since these recruitment positions are aimed at similar job seekers, who have similar user historical behavior data, the real recruitment effect can be restored based on the sampling ratio.

[0122] S22. Build a simulation system

[0123] The constructed simulation system may include a user context simulation module and an online environment simulation module. The user context simulation module and the online environment simulation module are described below respectively:

[0124] a) User context simulation module

[0125] i. Purpose: To restore the contextual information of real users requesting the recruitment platform app and accurately restore the contextual features required to execute the advertising promotion system. Specific contextual information includes user scenarios (specific tab pages, search or recommendation), user on-site behavior (search query, job information on similar job pages), entry source and its parameters (the previous page of the entry page, the parameters brought in), etc. Users can include recruiters and job seekers.

[0126] ii. Implementation method:

[0127] 1. Snapshot (i.e., save): User context features can be structured, saved through the recruitment platform app's log system, and returned to the log storage system. A unique identification ID is retained in each context log;

[0128] 2. Recovery: Extract context information from the recruitment platform app's log system, parse it, and generate a request URL for requesting the advertising promotion system in the simulation system;

[0129] b) Online environment simulation module

[0130] i. Purpose: To simulate the execution process of the advertising promotion system of a real recruitment platform app

[0131] ii. Implementation method:

[0132] 1. Snapshot storage: The recall and sorting models used every day record a version number and make a backup

[0133] 2. The positions to be evaluated are written into the ad database and indexed. The positions to be evaluated can be recalled using the following methods:

[0134] a) DSSM recall: Use the position to be evaluated and its related features to request the DSSM recall model. The recall model returns the position vector to be evaluated, which is written to ES for recall.

[0135] b) Function recall: the functions corresponding to the position to be evaluated are written into ES for recall;

[0136] c) Graph Embedding U2J recall: The job vector generated by the Graph Embedding model is written to ES for recall.

[0137] 3. DSSM recall model recovery: Restore the recall model with the corresponding version number based on the date of the sample data set. For example, when using the previous day's sample data to evaluate recruitment results, the recall model service of the previous day can be kept running without having to restore it every time.

[0138] Recovery process: Load the DSSM recall model into the model service and restore the corresponding configuration information.

[0139] 4. Sorting model recovery: restore the sorting model of the corresponding version number according to the date of the sample data set (same as the recall model, no need to restore it every time)

[0140] Recovery process: Load the sorting model into the model service and restore the corresponding configuration information

[0141] 5. System configuration file recovery

[0142] S23. Evaluation of delivery effect

[0143] S231. Determine the target ranking position of the position to be evaluated in the recommendation list

[0144] In the constructed simulation system, the following operations are performed on each sample data:

[0145] Restore the recommendation list generated based on each sample data from the simulation system log, and at the same time restore the ranking score of each position to be evaluated in different recommendation lists from the log.

[0146] S232. Determine the browsing depth of each sample data

[0147] Based on yesterday’s exposure log of the recruitment platform app, we determined the browsing depth of each sample data, that is, the number of positions that a job seeker saw in one browsing request.

[0148] S233. Estimate whether each sample data is exposed

[0149] S2331. Based on the ranking score of each position to be evaluated in different recommendation lists and the browsing depth of each corresponding sample data, determine whether the position to be evaluated is exposed under the sample data;

[0150] S2332. Calculate the delivery volume of the position to be evaluated after purchasing the advertising promotion service based on whether the position to be evaluated is exposed under each sample data, the estimated viewing rate of the position to be evaluated in the ranking model, and the estimated conversion rate from viewing to delivery.

[0151] Example 1

[0152] For example, determine the title of the position to be evaluated: Financial Accounting;

[0153] Functional positions to be assessed (Level 3 category): Accountant (Finance / Accountant);

[0154] The working city of the position to be evaluated: Shanghai.

[0155] Therefore, the target functional position grid for the position to be evaluated can be determined as "Shanghai Accountant".

[0156] S31. Use Graph Embedding to generate grid similarity and expand the "Shanghai Accountant" grid. The expanded grid may include: Shanghai Financial Analyst, Shanghai General Ledger Manager, Shanghai Financial Advisor, Shanghai Cost Manager, Shanghai Financial Specialist, Shanghai CPA, Shanghai Audit Manager / Supervisor, Shanghai Tax Specialist, Shanghai Audit Specialist, etc.

[0157] S32. Filter the sample data sets from the previous day within the current grid and similar grids in the recruitment platform app. Each sample data set corresponds to a different job seeker, and each sample data set includes multiple job positions viewed by the same job seeker. Each sample data set may also include contextual information about the job seeker's browsing experience, specifically including user profile data, job search expectations, browsing scenarios, server-retrieved jobs, sorted job lists, front-end exposed job lists, the job sequence viewed by the job seeker, clicked jobs, and submitted jobs.

[0158] For example, in sample data 1, the job positions browsed by job seeker Zhang San may include Shanghai accountant of Company A, Shanghai financial analyst of Company B, Shanghai general ledger supervisor of Company C, Shanghai accountant of Company D, etc.

[0159] S33. Use the constructed simulation system to simulate the online environment of the position to be evaluated after purchasing the advertising promotion service; wherein, the constructed simulation system can maintain a copy of the log system so that it does not need to be restored every time the position is estimated.

[0160] a) Restore yesterday’s online recall model and configuration in the recruitment platform app in the simulation system;

[0161] b) Restore yesterday’s online sorting model and configuration in the recruitment platform app in the simulation system;

[0162] c) Insert the job to be evaluated into the advertising library, that is, publish the job to be evaluated in the advertising promotion system.

[0163] S34. Restore the context feature data of sample data 1 in the constructed simulation system. An example is as follows:

[0164] a) The context parameters of sample data 1 include: user ID (resume ID), job function, time, user profile data, etc.

[0165] b) In sample data 1, the job seeker browsed 50 positions, which are recorded as JD_1_1, JD_1_2, JD_1_3, ... JD_1_50;

[0166] c) Set the job seeker browsing depth of sample data 1 to 50 positions;

[0167] S35. Simulate job recall in the constructed simulation system: Use the online recall model to recall the positions to be evaluated, and sort the positions based on the recall score and the recall results of sample data 1 in the recruitment platform app yesterday (recovered from the recall log system) to determine whether sample data 1 can be retained.

[0168] S36. Simulate ranking within the constructed simulation system: Use the online ranking model to score the positions to be evaluated. The refined ranking scores for the positions to be evaluated are placed into the position ranking queue for Sample Data 1 (recovered from the ranking log system) within the recruitment platform app for Sample Data 1 yesterday, and the ranking position is determined. If the position to be evaluated is discoverable in Sample Data 1, the online ranking model outputs the estimated view rate (pctr) and the estimated view-to-delivery conversion rate (pcvr).

[0169] S37. Determine whether the ranking position is within the browsing depth of sample data 1. If the ranking position is <= 50, the position to be evaluated, "Shanghai Accountant", can be exposed in sample data 1; otherwise, it will not be exposed.

[0170] S38. Repeat the above steps S34-S37 for each sample data in the sample data set filtered out in step S32.

[0171] S39. Traverse each sample data in the sample data set, count the total number of exposures yesterday, and obtain the total number of exposures for the position to be evaluated, "Financial Accounting."

[0172] S40. Calculate the delivery volume and ROI based on the total number of impressions and the total delivery rate for the position "Financial Accounting"; see the following conditional formula:

[0173] apply=N×∑pctr×pcvr strank≤depth

[0174]

[0175] Where: apply represents the number of resumes received for the position to be evaluated;

[0176] N represents the total number of exposures of the position to be evaluated;

[0177] ROI represents the number of resumes that can be obtained per unit cost;

[0178] Cost represents the cost of purchasing an advertising promotion service;

[0179] pctr represents the estimated viewing rate output by the ranking model;

[0180] pcvr represents the estimated view-to-delivery conversion rate output by the ranking model;

[0181] Rank indicates the ranking of the position to be evaluated in the position ranking queue;

[0182] depth indicates the browsing depth of job seekers; st indicates the constraint conditions.

[0183] In summary, compared with the existing technical solutions, the technical solution of this application can reduce the estimated delivery volume deviation by 65%.

[0184] In the embodiment of the present application, in order to save the cost of sample replay and reduce the system's computing time, combined with the characteristics of the recruitment industry, the target functional position grid is expanded using the similarity grid to simulate the real online position recall situation, so as to achieve the selectability of the replay samples. In this way, the system computing amount can be effectively reduced while ensuring the effect.

[0185] In the embodiments of this application, online environment simulation is achieved by saving and restoring snapshots of online dependent modules. Simulating user browsing behavior by saving and restoring the context of users browsing positions allows for an estimation of recruitment effectiveness. The ranking of the position to be evaluated in the target sample data and the user's actual browsing depth are combined to determine whether the position to be evaluated is exposed. The delivery volume is also calculated based on the position's delivery rate. Furthermore, log recovery is used to restore the recall and refined ranking queues, thereby obtaining the ranking position of the position to be evaluated, significantly improving system performance and execution efficiency.

[0186] The electronic device in the embodiment of the present application can be a user terminal device, a server, other computing devices, or a cloud server. Figure 4 A schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application is shown. The electronic device may include a processor 601 and a memory 602 storing computer program instructions. When the processor 601 executes the computer program instructions, the process or function of any of the above-mentioned embodiments is implemented.

[0187] Specifically, the processor 601 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application. The memory 602 may include a large-capacity memory for data or instructions. For example, the memory 602 may be at least one of the following: a hard disk drive (HDD), a read-only memory (ROM), a random access memory (RAM), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, a universal serial bus (USB) drive, or other physical / tangible memory storage device. For another example, the memory 602 may include a removable or non-removable (or fixed) medium. For another example, the memory 602 may be inside or outside the integrated gateway disaster recovery device. The memory 602 may be a non-volatile solid-state memory. In other words, the memory 602 typically includes a tangible (non-transitory) computer-readable storage medium (such as a memory device) encoded with computer-executable instructions, and when the software is executed (such as by one or more processors), the operations described in the method of the embodiments of the present application may be performed. The processor 601 implements the process or function of any of the methods in the above embodiments by reading and executing computer program instructions stored in the memory 602.

[0188] In one example, Figure 4 The electronic device shown may also include a communication interface 603 and a bus 610. The processor 601, memory 602, and communication interface 603 are connected via the bus 610 and communicate with each other. The communication interface 603 is primarily used to implement communication between the various modules, devices, units, and / or equipment in the embodiments of the present application. The bus 610 includes hardware, software, or both, and can couple the components of the online data traffic metering device to each other. For example, the bus may include at least one of the following: an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industrial Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Area Bus (VLB) bus, or other suitable buses. The bus 610 may include one or more buses. Although the embodiments of the present application describe or illustrate a specific bus, the embodiments of the present application may consider any suitable bus or interconnection method.

[0189] In combination with the method in the above embodiments, an embodiment of the present application also provides a computer-readable storage medium, which stores computer program instructions. When the computer program instructions are executed by a processor, they implement the process or function of any method in the above embodiments.

[0190] In addition, an embodiment of the present application further provides a computer program product, which stores computer program instructions. When the computer program instructions are executed by a processor, the process or function of any one of the methods in the above embodiments is implemented.

[0191] The flowcharts and / or block diagrams of the methods, devices, systems and computer program products of the embodiments of the present application are described above by way of example, and various aspects thereof are described. It should be understood that each box in the flowchart and / or block diagram or a combination thereof may be implemented by computer program instructions, or may be implemented by dedicated hardware that performs a specified function or action, or may be implemented by a combination of dedicated hardware and computer instructions. For example, these computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to form a machine that enables these instructions executed by such a processor to enable the implementation of the functions / actions specified in each box in the flowchart and / or block diagram or a combination thereof. Such a processor may be a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit.

[0192] The functional blocks shown in the block diagram of the embodiment of the present application can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc.; when implemented in software, it is a program or code segment used to perform the required task. The program or code segment can be stored in a memory or transmitted on a transmission medium or a communication link via a data signal carried in a carrier. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0193] It should be noted that the present application is not limited to the specific configurations and processes described above or shown in the figures. The above is only a specific implementation method of the present application. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the described system, device, module or unit can refer to the corresponding process in the method embodiment without further description. It should be understood that the scope of protection of the present application is not limited to this. Any technician familiar with the technical field can think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the scope of protection of the present application.

Claims

1. A recruitment effect estimation method, characterized in that: include: Obtain positions with the same or similar functional information as the position to be evaluated to obtain an expanded position set; Obtaining a sample data set of the positions to be evaluated and the expanded position set within a preset time period on the recruitment platform; wherein each sample data in the sample data set corresponds to a different job seeker, and each sample data includes multiple position data browsed by the same job seeker; Executing a recommendation request for the position to be evaluated in a pre-built simulation system to obtain a ranking score of the position to be evaluated in a recommendation list; wherein the simulation system includes a user context simulation module and an online environment simulation module; the user context simulation module is used to extract context feature data of each job seeker visiting the recruitment platform from the recruitment platform's log system; the online environment simulation module is used to execute a recommendation request for the position to be evaluated based on the context feature data corresponding to each job seeker, to obtain a ranking score of the position to be evaluated in the recommendation list of different job seekers; Sort the multiple positions of each sample data in the sample data set according to the scores of the first sorting model to obtain multiple position sorting queues; wherein each position sorting queue corresponds to each sample data one by one; According to the ranking scores of the positions to be evaluated in the recommendation lists of different job seekers, the positions to be evaluated are placed in a plurality of the position ranking queues to determine whether the positions to be evaluated can be exposed in each sample data; Count the number of times the position to be evaluated is exposed in the sample data set to determine the number of applications for the position to be evaluated.

2. The method according to claim 1, characterized in that The online environment simulation module includes a first recall model and a second ranking model; executing a recommendation request for the position to be evaluated in the pre-built simulation system to obtain a ranking score of the position to be evaluated in the recommendation list includes: Input the context feature data of the current sample data into the online environment simulation module, and use the first recall model to perform recall scoring on the position to be evaluated to obtain a first recall score; Obtaining a job recall queue of current sample data in a log system of the recruitment platform; wherein the job recall queue of the current sample data is a sorted queue obtained by respectively performing recall scoring on multiple jobs browsed by the current job seeker according to a second recall model of the recruitment platform, and sorting the jobs according to the recall scores; Based on the first recall score and the job recall queue of the current sample data, the matching degree between the current job seeker and the job to be evaluated is determined; If the matching degree between the current job seeker and the position to be evaluated meets the preset conditions, the current sample data will be retained; Executing a recommendation request for the position to be evaluated in the context feature data of the retained current sample data to generate a recommendation list for the current job seeker; The recommendation list of the current job seeker is sorted and scored using the second sorting model to obtain a sorting score of the position to be evaluated in the recommendation list of the current job seeker.

3. The method according to claim 2, characterized in that The process of placing the positions to be evaluated in a plurality of position ranking queues according to the ranking scores of the positions to be evaluated in the recommendation lists of different job seekers, so as to determine whether the positions to be evaluated can be exposed in each sample data, includes: Determine the target ranking position of the position to be evaluated in the position ranking queue of the current sample data based on the ranking score of the position to be evaluated in the recommendation list of the current job seeker and the position ranking queue of the current sample data; If the target ranking position is within the preset browsing depth of the current sample data, the position to be evaluated is determined to be exposed in the current sample data; Traverse each sample data in the sample data set to obtain the total number of times the position to be evaluated is exposed in the sample data set.

4. The method according to claim 1, wherein The context feature data of the job seeker when visiting the recruitment platform includes one or more of the following: resume ID, job position, time, user portrait data, user behavior scenario data and page entry source parameters.

5. The method according to claim 3, characterized in that The method of counting the number of exposures of the position to be evaluated in the sample data set to determine the number of applications for the position to be evaluated includes: When it is determined that the position to be evaluated is exposed in the current sample data, the current estimated viewing rate and the current estimated conversion rate of the position to be evaluated output by the second ranking model are obtained; The current delivery rate is calculated based on the current estimated view rate and the current estimated view-to-delivery conversion rate; Traverse each sample data exposed in the sample data set to obtain the total delivery rate; The delivery volume for the position to be evaluated is calculated based on the total delivery rate and the sum of the exposure times.

6. The method according to claim 1, characterized in that Also includes: Screen out high-potential positions from multiple candidate positions and use them as positions to be evaluated; among them, High-potential positions include positions with a staffing ratio below a set threshold, positions that are in urgent need of recruitment, or positions with a high probability of purchasing value-added services predicted by the model.

7. The method according to any one of claims 1 to 6, characterized in that The step of obtaining positions having the same or similar functional information as the position to be evaluated to obtain an expanded position set includes: Obtaining the functional information of the position to be evaluated, and expanding the functional information of the position to be evaluated to obtain similar functional information; Carry out multi-level functional classification of the positions to be evaluated to obtain the target functional positions; Combine the target functional position with the regional information to obtain a target functional position grid; wherein the functional information of the position to be evaluated includes regional information; Through the graph embedding algorithm, the target functional position grid is expanded using the similarity grid to obtain an extended position set; wherein the similarity grid is the grid where positions with the same or similar functional information as the position to be evaluated are located.

8. An electronic device, characterized in that: The electronic device comprises: a processor and a memory storing computer program instructions; and when the electronic device executes the computer program instructions, the method according to any one of claims 1 to 7 is implemented.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that The computer program product includes computer program instructions, and when the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images