Generation method and device of occupational planning large model, electronic equipment and storage medium

Abstract

The invention discloses a generation method and device of an occupational planning large model, electronic equipment and a storage medium. The method comprises the steps that a first training data set is acquired, the first training data set comprises occupational research information and desensitized resume information and position information, and the occupational research information comprises corresponding relation information of occupations and functions; pre-training the general large model by using the first training data set and an autoregression method to obtain a recruitment field large model; and generating an instruction fine tuning training data set based on the information of the plurality of job seekers and the expert occupational planning suggestions corresponding to the information of each job seeker. According to the embodiment of the invention, the large occupational planning model can be generated, and the occupational planning model can output professional occupational planning suggestions according to the input job seeker information.

Description

Technical Field

[0001] The present application relates to the field of generative artificial intelligence technology, specifically to the field of model training, model fine-tuning and natural language processing technology, and in particular to a method, device, electronic device, computer-readable storage medium and computer program product for generating a large career planning model. Background Art

[0002] In today's information age, college students and new employees struggle to fully access information about different occupations and industries, making it difficult to develop a career plan that suits them. Common solutions include: First, seeking help from professional career planning experts. For example, users provide their personal information to experts or career planners in their field of interest. These experts, drawing on their experience, provide personalized career planning advice tailored to the user's specific situation. With the help of career planners, they can develop a career plan that suits them. Second, career planning can be derived based on big data statistics or existing large language models.

[0003] However, both of the above methods have drawbacks. The first solution is to provide job seekers with expert career planning advice. For third-party agencies providing recruitment services, this requires support from career planning experts in various categories and fields to achieve a certain coverage rate. The labor cost is undoubtedly high, and for each job seeker, the cost of using expert private career planning services is relatively high and not practical. The second method is to develop career plans for job seekers based on Internet big data. Now you can get answers through the entrance of the generative big model, but the practical results are not satisfactory, and it cannot be well customized, which is far from user expectations. Summary of the Invention

[0004] In view of this, embodiments of the present application provide a method, device, electronic device, computer-readable storage medium, and computer program product for a career planning model to solve at least one technical problem.

[0005] The embodiment of the present application provides a method for generating a career planning big model, comprising: obtaining a first training data set, wherein the first training data set includes career research information and resume information and position information that have been desensitized, wherein the career research information includes information on the correspondence between careers and functions; using the first training data set, using an autoregressive method to pre-train a general big model to obtain a recruitment field big model; generating an instruction fine-tuning training data set based on multiple job seeker information and expert career planning suggestions corresponding to each job seeker information; using the instruction fine-tuning training data set to perform instruction fine-tuning training on the recruitment field big model to obtain a career planning big model; inputting the desensitized information of multiple target job seekers into the career planning big model to obtain multiple groups of expert career planning information output by the career planning big model, wherein each group of expert career planning information includes multiple expert career planning suggestions randomly sorted; The career planning information is labeled, and the multiple career planning suggestions in each group of expert career planning information after labeling are sorted by quality; a second training data set is formed based on the multiple target job seeker information and the labeled expert career planning information; according to the quality ranking, the training data in the second training data set are input into the general big model in turn, and a reward model is obtained after the training is completed, wherein the loss function used in the training is based on the difference between the output value of the reward model corresponding to the first input data and the output value of the reward model corresponding to the later input data; the career planning big model includes a career planning big model with fixed parameters and a career planning big model with fine-tuning parameters, and a first expectation function is constructed based on the reward model and the career planning big model with fixed parameters; the career planning big model with fine-tuning parameters is trained based on the target job seeker information and the first expectation function, and a first version of the career planning big model with updated parameters is obtained after the training is completed.

[0006] Optionally, according to the method of an embodiment of the present application, each job seeker information in the multiple job seeker information includes at least one personal information of the job seeker; generating an instruction fine-tuning training data set based on the multiple job seeker information and the expert career planning advice corresponding to each job seeker information further includes: splicing at least one personal information in each job seeker information into a long text; embedding the long text and the corresponding career planning advice into a corresponding generation template, wherein the generation template includes an instruction part and an output value part, the instruction part is used to write the long text, and the output value part is used to write the corresponding expert career planning advice; generating the instruction fine-tuning training data set required for instruction fine-tuning through the generation template.

[0007] Optionally, according to the method of an embodiment of the present application, the at least one personal information of the job seeker includes at least one of the following: the job seeker's skill status information, the job seeker's living status information, the job seeker's personality assessment information, and the job seeker's future expectation information.

[0008] Optionally, according to the method of the embodiment of the present application, the loss function is:

[0009]

[0010] Among them, y w and y l Both represent career planning, w is sorted in y l Previous career planning, x is the prompt word from the second training data set, r θ (x,y w ) is the reward model output value corresponding to the first input data, r θ (x,y l ) is the output value of the reward model corresponding to the subsequent input data, σ is the coefficient, and K is the number of career planning suggestions corresponding to the same target job seeker information.

[0011] Optionally, according to the method of an embodiment of the present application, the parameter-fine-tuned career planning model is trained based on the target job seeker information and the first expected function, including: scoring the career planning output by the parameter-fine-tuned career planning model based on the reward model, and constraining the degree of deviation of the parameters of the parameter-fine-tuned career planning model based on the fixed-parameter career planning model.

[0012] Optionally, according to the method of the embodiment of the present application,

[0013] The first expectation function is:

[0014]

[0015] Among them, x RL is the prompt word from the training data of the career planning model with fine-tuned training parameters, y is the output word, and RM represents the reward model LLM SFI Represents a large model of career planning with fixed parameters; is the first version of the career planning model, parameterized by φ, β is the coefficient, Represents the KL divergence loss of the career planning model with fixed parameters and the first version of the career planning model.

[0016] Optionally, the method according to the embodiment of the present application further includes:

[0017] Based on the perplexity function, a constraint function is added to the first expectation function to obtain a second expectation function, and based on the second expectation function, the career planning model with fine-tuned parameters is trained. After the training is completed, a second version of the career planning model with updated parameters is obtained; wherein,

[0018] The constraint function is:

[0019]

[0020] The second expectation function is:

[0021]

[0022] Where x is the prompt word from the training data of the career planning model with fine-tuned training parameters, x pretrain is the prompt word RM(x RL，y ) is the preference loss of the reward model, and the perplexity function is negative It is the KL divergence loss between the career planning model with fixed parameters and the career planning model with fine-tuned parameters.

[0023] The embodiment of the present application also proposes a device for generating a career planning big model, including: an acquisition module for acquiring a first training data set, wherein the first training data set includes career research information and resume information and position information that have been desensitized, wherein the career research information includes information on the correspondence between careers and functions; a pre-training module for using the first training data set to pre-train a general big model using an autoregressive method to obtain a recruitment field big model; a generation module for generating an instruction fine-tuning training data set based on multiple job seeker information and expert career planning suggestions corresponding to each job seeker information; an instruction fine-tuning training module for using the instruction fine-tuning training data set to perform instruction fine-tuning training on the recruitment field big model to obtain a career planning big model; a first input module for inputting multiple target job seeker information that has been desensitized into the career planning big model to obtain multiple groups of expert career planning information output by the career planning big model, wherein each group of expert career planning information includes multiple expert career planning suggestions that are randomly sorted; a labeling module for The career planning information of each group of experts is labeled, and the multiple career planning suggestions in the labeled career planning information of each group of experts are sorted by quality; a second training data set forming module is used to form a second training data set based on the multiple target job seeker information and the labeled expert career planning information; a second input module, according to the quality ranking, sequentially inputs the training data in the second training data set into the general large model, and obtains a reward model after the training is completed, wherein the loss function used in the training is based on the difference between the output value of the reward model corresponding to the first input data and the output value of the reward model corresponding to the later input data; a construction module, the career planning large model includes a career planning large model with fixed parameters and a career planning large model with fine-tuning parameters, and the construction module is used to construct a first expectation function based on the reward model and the career planning large model with fixed parameters; a training module is used to train the career planning large model with fine-tuning parameters based on the target job seeker information and the first expectation function, and obtain the first version of the career planning large model with updated parameters after the training is completed.

[0024] An embodiment of the present application provides an electronic device, which includes a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the steps of the method described above are implemented.

[0025] An embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the steps of the method described above are implemented.

[0026] An embodiment of the present application provides a computer program product, which includes computer program instructions. When the computer program instructions are executed by a processor, the steps of the method described above are implemented.

[0027] By using the embodiments of the present application, a large model of career planning in the recruitment field can be generated. Due to specific fine-tuning, the career planning model can output professional career planning suggestions based on the input job seeker information. For the majority of users, it can replace career planning experts to a certain extent and obtain tailor-made planning suggestions. Therefore, the large model of career planning based on the present application can provide highly professional expert career planning suggestions to a wider range of users, which is not only conducive to reducing costs, but also can provide services to job seekers of various categories, fields and levels, which can fundamentally improve the coverage of career planning services and maximize convenience for users. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings in the embodiments of the present application.

[0029] Figure 1 It is a schematic diagram of the system architecture of an embodiment of the present application.

[0030] Figure 2 It is a flowchart of the method for generating a career planning model in an embodiment of the present application.

[0031] Figure 3 It is a schematic diagram of multiple input texts of the second training data set of the career planning model in an embodiment of the present application.

[0032] Figure 4 This is a diagram illustrating the training process of the general large model in the pre-training stage of an embodiment of the present application.

[0033] Figure 5 It is a schematic diagram of the process of generating text from the large model in the recruitment field according to an embodiment of the present application.

[0034] Figure 6 It is a schematic diagram of the training process of the large recruitment model in the instruction fine-tuning training stage of an embodiment of the present application.

[0035] Figure 7 This is a flowchart of obtaining reward model training samples in an embodiment of the present application.

[0036] Figure 8 Schematic diagram of the training process of the reward model of the embodiment of the present application.

[0037] Figure 9 It is a schematic diagram of the training process of the career planning model with fine-tuning of parameters in an embodiment of the present application.

[0038] Figure 10 This is a schematic diagram of the interaction using the career planning model in an embodiment of the present application.

[0039] Figure 11It is a structural block diagram of a device for generating a large career planning model according to an embodiment of the present application.

[0040] Figure 12 It is a schematic diagram of the hardware structure of the electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0041] The principles and spirit of the present application will be described below with reference to several exemplary embodiments. It should be understood that the purpose of providing these embodiments is to make the principles and spirit of the present application clearer and more thorough, so that those skilled in the art can better understand and implement the principles and spirit of the present application. The exemplary embodiments provided herein are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments herein, all other embodiments obtained by those of ordinary skill in the art without creative work are within the scope of protection of this application.

[0042] The embodiments of the present application relate to terminal devices and / or servers. Those skilled in the art will appreciate that the embodiments of the present application may be implemented as a system, apparatus, device, method, computer-readable storage medium, or computer program product. Therefore, the present application may be specifically implemented in at least one of the following forms: complete hardware, complete software, or a combination of hardware and software. According to the embodiments of the present application, the present application seeks protection for a method, apparatus, electronic device, computer-readable storage medium, and computer program product for generating a large career planning model. Figure 1 1 is a schematic diagram of a system architecture of an embodiment of the present application, which includes a terminal device 102 and a server 104. The terminal device 102 may include at least one of the following: a smartphone, a tablet computer, a laptop computer, a desktop computer, a smart TV, various wearable devices, an augmented reality (AR) device, a virtual reality (VR) device, and the like. A client may be installed on the terminal device 102. For example, the client may be a client that specifically performs a specific function (such as an application app), or a client that embeds multiple application applets (with different functions), or a client that logs in through a browser. The user can operate on the terminal device 102. For example, the user can open the client installed on the terminal device 102 and input instructions through the client operation, or the user can open the browser installed on the terminal device 102 and input instructions through the browser operation. After the terminal device 102 receives the instruction input by the user, it sends a request message containing the instruction to the server 104. After receiving the request message, the server 104 performs the corresponding processing and then returns the processing result information to the terminal device 102. The user instruction is completed through a series of data processing and information interaction.

[0043] In this document, terms such as first, second, and third are only used to distinguish one entity (or operation) from another entity (or operation), and are not intended to require or imply any order or relationship between these entities (or operations).

[0044] The following is a brief description of the concepts, technical terms and other related contents that may be involved in the embodiments of this application.

[0045] Generative AI (Artificial Intelligence Generated Content, AIGC) is a type of artificial general intelligence (AGI) technology focused on generating or creating new content. It uses machine learning to generate new text, program code, images, videos, and sounds based on large-scale multimodal datasets. It can handle multiple tasks and be applied to multiple scenarios, such as text generation, text summarization, machine translation, and question-answering, and possesses high generalization capabilities. The application of AIGC technology improves the efficiency of content generation, saving time and costs for creators while expanding the possibilities of content creation. The fundamental technical support for AIGC is the Large Language Model (LLM), a language model containing billions or more parameters that generates human-like text by learning to understand large amounts of text and contextual data. Examples of LLMs include the Generative Pre-Trained Transformer (GPT), Baichuan2-7B, and Baichuan2-13B.

[0046] LLM undergoes comprehensive training, including pre-training, supervised fine-tuning (SFT), and reinforcement learning from human feedback (RLHF), enabling it to model and generate natural language text. During the pre-training phase, the model learns the language structure, grammar, and factual knowledge contained in a large amount of text data.

[0047] In the SFT stage, the pre-trained LLM is fine-tuned with labeled data to make it more suitable for performing a specific task. In the RLHF stage, reinforcement learning methods are used to directly optimize the language model using human feedback signals.

[0048] After receiving user input, the client processes it through the intent understanding module. This module determines whether to trigger Retrieval-Augmented Generation (RAG) to process the user input, effectively understanding the intent.

[0049] Figure 2 This is a flowchart of a method for generating a career planning model according to an embodiment of the present application, which method includes the following steps:

[0050] S101: Obtain a first training data set, the first training data set including career research information and desensitized resume information and position information, wherein the career research information includes information on the correspondence between careers and functions;

[0051] S102: Using the first training data set, pre-train the general large model using an autoregressive method to obtain a large model for the recruitment domain;

[0052] S103: Generate an instruction fine-tuning training dataset based on multiple job seeker information and expert career planning suggestions corresponding to each job seeker information;

[0053] S104: Using the instruction fine-tuning training dataset, perform instruction fine-tuning training on the recruitment domain model to obtain the career planning model.

[0054] S105: Inputting the desensitized information of the multiple target job seekers into the career planning model to obtain multiple groups of expert career planning information output by the career planning model, each group of expert career planning information including multiple randomly ordered expert career planning suggestions;

[0055] S106: Labeling the career planning information of each group of experts, and sorting the career planning suggestions in the labeled career planning information of each group of experts in order of merit;

[0056] S107: forming a second training data set based on the information of multiple target job seekers and the annotated expert career planning information;

[0057] S108: Input the training data in the second training data set into the general large model in sequence according to the order of merit. After the training is completed, a reward model is obtained, wherein the loss function used in the training is based on the difference between the output value of the reward model corresponding to the first input data and the output value of the reward model corresponding to the second input data;

[0058] S109: The career planning model includes a career planning model with fixed parameters and a career planning model with fine-tuned parameters, and a first expectation function is constructed based on the reward model and the career planning model with fixed parameters;

[0059] S110: Based on the target job seeker information and the first expectation function, train the career planning large model with fine-tuned parameters. After the training is completed, the first version of the career planning large model with updated parameters is obtained.

[0060] For the generation method of the career planning large model of this application, steps S101 and S102 correspond to the pre-training stage of the general large language model. By pre-training the general large language model, the general large language model can learn knowledge in the recruitment field. In the pre-training stage, the first training data set for training the general large model includes: desensitized resume information, desensitized job information, and career research information. Here, it should be noted that: a job refers to a work position. For example, management positions include: general manager, deputy general manager, human resources director, financial director, marketing director; a career refers to different types of jobs; the corresponding relationship information between a career and functions is, for example, for a programmer (career), the corresponding functions include: C language, JAVA, etc.

[0061] After having the first training data set, train the large model in an autoregressive manner. The basic idea of an autoregressive model is to represent a complex sequence as a simple conditional probability distribution, that is, given past information, calculate the probability distribution of the next element. Specifically, for a sequence x1, x2, …, x T , the autoregressive model believes that when x1 to x T-1 are known, the conditional probability distribution of x T can be represented by P(x T ∣ x1, x2, …, x T-1 ). During the training process, the autoregressive model learns the conditional probability distribution by maximizing the log-likelihood function. By adjusting the parameters of the general large model, the probability of the next element predicted by the model is maximized, so as to learn the statistical laws in the data. For example, in real life, for the sentence "Knowledge is power", through continuous training and continuous adjustment of the model's parameters, the conditional probability distribution of "shi" after "zhi" given by the model is 80% in the end, then this model can generate certain text. In the embodiment of this application, the general large model is pre-trained with the input first training data to obtain a recruitment field large model, and the recruitment field large model learns knowledge related to the recruitment field.

[0062] In the embodiment of this application, S103 and S104 are the instruction fine-tuning training stage of the recruitment field large model. Instruction fine-tuning training is a further sub-training of the recruitment field large model obtained in the pre-training stage, so that the trained career planning large model can: give career planning suggestions according to the input job seeker information.

[0063] During the instruction fine-tuning training phase, the instruction fine-tuning training dataset includes multiple job seeker information and expert career planning suggestions corresponding to each job seeker information. In some embodiments of the present application, the instruction fine-tuning training dataset optionally includes multiple pairs of job seeker information and corresponding expert career planning suggestions, wherein the job seeker information in the multiple pairs of job seeker information and corresponding expert career planning suggestions can be the same, while the expert career planning suggestions are different. For example:

[0064] Job seeker a's information + career plan A;

[0065] Job seeker a information + career plan B;

[0066] Job seeker a information + career plan C.

[0067] The recruitment domain model trained using the instruction-based fine-tuning training dataset is called the career planning model. After training, the recruitment domain model learns the relationship between job seeker information and career planning recommendations from experts. Once trained, the career planning model can provide career planning recommendations when fed with job seeker information. For example, given job seeker A's information (name, personality, age, degree, etc.), the career planning model outputs position plan A, position plan B, or career plan C.

[0068] In the embodiment of the present application, steps S105 to S110 are the reinforcement learning phase of the career planning model. The purpose of this phase is to make the output of the career planning model more accurate. In this phase, a reward model is used to score the output structure of the career planning model. Specifically, a reward model is first trained using steps S105 to S108 to obtain a reward model that can score the expert career planning suggestions output by the career planning model.

[0069] During the training process of the reward model, training data must be obtained first. In order to obtain training data, the career planning model must first input the desensitized resume information of the applicant (target job seeker information) so that the career planning model can output multiple expert career planning suggestions. For example, given the information of job seeker a, the career planning model outputs career planning A or position planning B or career planning C. Then let the career planning expert rank career planning A, career planning B, and career planning C. The result is career planning A>career planning B>career planning C. Then construct a second training data set, which includes multiple input texts, each of which includes job seeker information and expert career planning information output by the career planning model (for example, attached Figure 3The second training data set is then input into the general large model. The order of inputting the training data follows the order of expert ranking, for example, text A is input first, then text B. Furthermore, when constructing the loss function of the constrained reward model, the loss function is constructed based on the difference between the reward model output value corresponding to the first input data and the reward model output value corresponding to the second input data.

[0070] In some embodiments of the present application, optionally, the samples in the second training data may be identified as positive samples and negative samples so that the value of the loss function is the difference between the reward score corresponding to the positive sample and the reward score corresponding to the negative sample.

[0071] Furthermore, based on the loss function, the parameters of the general large model are continuously optimized. When the loss function is minimum, the parameters of the general large model are the parameters of the reward model. The obtained reward model can give high scores to the career planning suggestions ranked first and low scores to the career planning suggestions ranked last.

[0072] Next, the career planning model undergoes intensive training. This involves setting up a career planning model with adjustable parameters and a career planning model with fixed parameters. The career planning model with fixed parameters is called the career planning model with fixed parameters, while the career planning model with adjustable parameters is called the career planning model with fine-tuned parameters. Based on the first expectation function, the reward model is used to score the fine-tuned career planning model. The reward model's score for the fine-tuned career planning model serves as part of the first expectation function, while the other part is the KL divergence loss between the fixed-parameter career planning model and the first version of the career planning model. During the process of adjusting the parameters of the fine-tuned career planning model, the value of the first expectation function continuously changes. When the reward model's score is highest and the KL divergence loss is lowest, the value of the first expectation function is maximized. At this point, the parameters of the fine-tuned career planning model are the same as those of the first version of the career planning model.

[0073] The present embodiment of the present invention develops a first version of a career planning model by training a large model using multiple training data sets. This career planning model can output professional career planning suggestions based on the input job seeker information. Therefore, the career planning model proposed in this application can replace career planning experts, solving the fundamental problem of human resource limitations, thereby significantly reducing prices while increasing the coverage of career planning services.

[0074] In some embodiments of the present application, optionally, each job seeker information in the plurality of job seeker information includes at least one personal information of the job seeker;

[0075] Generating an instruction fine-tuning training dataset based on multiple job seeker information and expert career planning suggestions corresponding to each job seeker information further includes:

[0076] Splice at least one piece of personal information in each job applicant's information into a long text;

[0077] Embed the long text and the corresponding career planning suggestions into the corresponding generation template, wherein the generation template includes an instruction part and an output value part. The instruction part is used to write the long text, and the output value part is used to write the corresponding expert career planning suggestions;

[0078] Generate the instruction fine-tuning training dataset required for instruction fine-tuning by generating a template.

[0079] In an embodiment of the present application, each job seeker information includes at least one item of personal information of the job seeker. For example, for multiple job seeker information (A job seeker information, B job seeker information, and C job seeker information), the personal information included in each job seeker information (A job seeker information) may be: the job seeker's skill status (that is, the job seeker's skill level, for example, job seeker A's programming level is very high).

[0080] Correspondingly, the instruction fine-tuning training dataset includes multiple texts, each of which consists of job seeker information and corresponding expert career planning suggestions. In some embodiments of the present application, each text is optionally composed of one job seeker information and one corresponding expert career planning suggestion. For example, the instruction fine-tuning training dataset includes:

[0081] Job seeker a information + expert career planning advice A;

[0082] Job seeker information a + expert career planning advice B;

[0083] Job seeker a information + expert career planning advice C;

[0084] Job seeker b information + expert career planning advice D;

[0085] Job seeker b information + expert career planning advice E;

[0086] …

[0087] In this application, before constructing a training dataset for fine-tuning instructions, the job applicant's information and at least one personal item must be concatenated into a long text. This long text corresponds to the resume information described above. The instructions and output values ​​in the growth template can then be mapped to the long text and expert career planning advice, respectively. This allows the system to embed the concatenated long text and the corresponding expert career planning advice into the generated template, quickly generating the required training data for fine-tuning instructions.

[0088] In some embodiments of the present application, optionally, the at least one personal information of the job applicant includes at least one of the following: the job applicant's skill status information, the job applicant's living status information, the job applicant's personality assessment information, and the job applicant's future expectation information.

[0089] In the embodiment of the present application, personal information may also include educational background information, work experience information, etc. Such personal information can be used to enrich the training data, so that the large model can provide more accurate career planning suggestions.

[0090] In some embodiments of the present application, optionally, the loss function is:

[0091]

[0092] Among them, y w and y l Both represent career planning, w is sorted in y l Previous career planning, x is the prompt word from the second training data set, r θ (x,y w ) is the reward model output value corresponding to the first input data, r θ (x,y l ) is the output value of the reward model corresponding to the subsequent input data, σ is the coefficient, and K is the number of career planning suggestions corresponding to the same target job seeker information.

[0093] In this application, the loss function is used to constrain the reward model. When the value of the loss function is minimized, the corresponding parameters of the general large model are the parameters of the reward model. For the training data of the reward model, the text entered first corresponds to y w , the text entered after corresponds to y t When the loss function is minimized, the difference between the scores of the first training data and the last training data is the largest, indicating that the reward model can give high scores to the career plans that are ranked first and low scores to the career plans that are ranked last.

[0094] In some embodiments of the present application, optionally, the parameter-fine-tuned career planning model is trained based on the target job seeker information and the first expected function, including: scoring the career planning output by the parameter-fine-tuned career planning model based on the reward model, and constraining the degree of deviation of the parameters of the parameter-fine-tuned career planning model based on the fixed-parameter career planning model.

[0095] During the training stage of the first version of the career planning model, it is necessary to continuously adjust the parameters of the career planning model with fine-tuning parameters based on the first expectation function. The first expectation function includes a part constructed based on the reward model and a part constructed based on the career planning model with fixed parameters. The part corresponding to the reward model is the score given to the career planning by the reward model, and the part corresponding to the career planning model with fixed parameters is used to constrain the parameters of the career planning model with fine-tuning parameters to not deviate too much from the parameters of the career planning model obtained during instruction fine-tuning.

[0096] By using the reward model and the career planning model with fixed parameters, the parameters of the career planning model with fine-tuning parameters can be adjusted, so that the first version of the career planning model can output high-quality career planning suggestions.

[0097] In some embodiments of the present application, optionally, the first expected function is:

[0098]

[0099] Among them, x RL is the prompt word from the training data of the career planning model with fine-tuned training parameters, y is the output word, and RM represents the reward model LLM SFI Represents a large model of career planning with fixed parameters; is the first version of the career planning model, parameterized by φ, β is the coefficient, Represents the KL divergence loss of the career planning model with fixed parameters and the first version of the career planning model.

[0100] In the first expectation function, RM(x RL , y) represents the score given by the reward model to the expert's career planning suggestions. For example, if the job seeker a and the corresponding career planning suggestion A are input into the reward model, the reward model will give a reward score M. If the job seeker a and the corresponding career planning suggestion N are input into the reward model, M>N. The KL divergence loss is the difference between the fixed-parameter career planning model and the first-version career planning model. This KL divergence loss is used to prevent the parameters of the fine-tuned career planning model from deviating too far from the parameters of the career planning model obtained during instruction fine-tuning when training the first-version career planning model.

[0101] In some embodiments of the present application, optionally, the method further includes:

[0102] Based on the perplexity function, a constraint function is added to the first expectation function to obtain a second expectation function. Based on the second expectation function, the career planning model with fine-tuned parameters is trained. After the training is completed, the second version of the career planning model with updated parameters is obtained.

[0103] The constraint function is:

[0104]

[0105] The second expectation function is:

[0106]

[0107] Where x is the prompt word from the training data of the career planning model with fine-tuned training parameters, x pretrain is the prompt word from the first training data set, RM(x RL ,y) is the preference loss of the reward model, and the perplexity function is negative It is the KL divergence loss between the career planning model with fixed parameters and the career planning model with fine-tuned parameters.

[0108] In an embodiment of the present application, when training the large career planning model with fine-tuning parameters, a constraint function can be added on the basis of the first expected function, so as to constrain the generated content of the large career planning model with fine-tuning parameters to not be too poor.

[0109] The above describes the implementation of the embodiments of the present application and the advantages brought by them through multiple embodiments. The following describes the specific processing process of the embodiments of the present application in detail with reference to specific examples.

[0110] Phase 1: Training the general large model so that it can learn knowledge in the recruitment field (understanding the terminology in the recruitment field)

[0111] Combine Figure 4 As shown, the model is trained using the internally desensitized resume content, job content, and career research content through autoregressive method. The formula corresponding to the autoregressive method is as follows:

[0112]

[0113] Combine Figure 5 As shown in the figure, the large recruitment domain model obtained after training can predict the probability of the next word based on the previous text content and call it to the end of the text in sequence, thereby generating text content related to the recruitment field.

[0114] Phase 2: Instruction fine-tuning (further subdividing the large recruitment model trained in the previous phase to enable it to provide career planning advice)

[0115] Combine Figure 6As shown, training data is constructed using job seeker information (e.g., job seeker resume information) and the corresponding career planning advice from career planning experts. During the construction process, a template can be generated. By mapping the job seeker resume and the corresponding career planning advice to the instruction part and output part of the growth model, multiple training data can be obtained. For example, one of the training data is:

[0116] Resume information of job seeker A + career plan A

[0117] Resume information of job seeker A + career plan B

[0118] Resume information of job seeker A + career plan C

[0119] The model trained in this step is called the Career Planning Master Model. It learns the relationship between a candidate's resume information and the career planning recommendations provided by experts. After training, if the model is fed with resume information, it can provide career planning recommendations. For example, given information A, it outputs position plan A, position plan B, or career plan C.

[0120] Phase 3: Obtaining Reward Model Samples

[0121] Combine Figure 7 As shown, the resume information is input into the career planning model obtained in the previous stage to obtain multiple career planning suggestions (position planning A, position planning B, career planning C). Then, career planning experts can be invited to rank these career planning opinions according to their merits, and the result is career planning A>career planning B>career planning C.

[0122] Stage 4: Training the Reward Model

[0123] Combine Figure 8 As shown, first construct the training data, which includes multiple input texts. The text content refers to Figure 2 shown.

[0124] The training data is then input into the input text in order according to the expert ranking, and a setting is given to the general large model, the first input corresponds to yw in the loss function, and the later input corresponds to yt in the loss function.

[0125] Among them, the loss function is as follows:

[0126]

[0127] Then let the model adjust the parameters according to this loss function. θ (x, yt) represents the mathematical formula corresponding to the reward model, where θ is the parameter of the reward model, and the output of the reward model is the score.

[0128] During model training, the loss must be minimized. To minimize loss, the reward score for the first training data input must be significantly greater than the reward score for the last training data input. After training, the reward model assigns higher scores to career plans before sorting and lower scores to career plans after sorting.

[0129] Phase 5: The career planning model adjusts its output based on the expert rankings;

[0130] Combine Figure 9 As shown, based on the expectation function, the career planning model with reward model and parameter anchoring is used ( Figure 9 The AI ​​career planning model (fixed version) and the constraint function (negative perplexity function) in the career planning model with fine-tuning parameters (AI career planning model (fine-tuning version)) are used to adjust the parameters of the career planning model with fine-tuning parameters, so that the output of the career planning model conforms to human preferences.

[0131] The expected function is as follows:

[0132]

[0133] Where x is the prompt word from the training data of the career planning model with fine-tuned training parameters,

[0134] RM(x RL , y) is the preference loss of the reward model, and the perplexity function is negative It is the KL divergence loss between the career planning model with fixed parameters and the career planning model with fine-tuned parameters.

[0135] During the training process, the resume information of the job seeker (target job seeker information) is input into the parameter-adjusted career planning model. Then, the expectation function is used to continuously adjust the parameters of the parameter-adjusted model in the process of maximizing the value of the expectation function. Finally, the final career planning model is obtained, that is, Figure 9 The AI ​​career planning model (fine-tuned version) with updated parameters shown.

[0136] Figure 10 This is a flowchart of the use of a career planning model in the embodiment of this application. Figure 10 As shown, in order to facilitate interaction with users, an AI digital human image can be designed for the final career planning model. The image of the AI ​​digital human can be generated and designed based on the user information already available on the platform. Users can complete information collection step by step by interacting with the AI ​​digital human, and the career planning model will provide career planning suggestions that match the user and are similar to the preferences of career planning experts. These career planning suggestions can be fed back to users through voice or PDF reports.

[0137] Corresponding to the method embodiment of the present application, the present application also provides a device for generating a career planning model, such as Figure 11 As shown, the career planning model device 100 includes:

[0138] An acquisition module 110 is configured to acquire a first training data set, the first training data set including career research information and desensitized resume information and position information, wherein the career research information includes information on the correspondence between careers and functions;

[0139] A pre-training module 120 is configured to use the first training data set and an autoregressive method to pre-train the general large model to obtain a large model for the recruitment domain;

[0140] A generating module 130 is configured to generate an instruction fine-tuning training dataset based on information of multiple job seekers and expert career planning suggestions corresponding to each job seeker information;

[0141] An instruction fine-tuning training module 140 is configured to perform instruction fine-tuning training on the recruitment domain model using an instruction fine-tuning training dataset to obtain a career planning model.

[0142] The first input module 150 is used to input the desensitized information of multiple target job seekers into the career planning model, and obtain multiple sets of expert career planning information output by the career planning model, each set of expert career planning information including multiple randomly ordered expert career planning suggestions;

[0143] A marking module 160 is used to mark the career planning information of each group of experts, and sort the career planning suggestions in the marked career planning information of each group of experts according to their merits;

[0144] A second training data set forming module 170 is configured to form a second training data set based on the target job seekers' information and the annotated expert career planning information;

[0145] A second input module 180 sequentially inputs the training data in the second training data set into the general large model according to the ranking of merit, and obtains a reward model after training, wherein the loss function used in the training is based on the difference between the output value of the reward model corresponding to the first input data and the output value of the reward model corresponding to the second input data;

[0146] A construction module 190, wherein the career planning model includes a career planning model with fixed parameters and a career planning model with fine-tuned parameters, and the construction module is used to construct a first expectation function based on the reward model and the career planning model with fixed parameters;

[0147] The training module 210 is used to train the career planning model with fine-tuned parameters based on the target job seeker information and the first expected function, and obtain a first version of the career planning model with updated parameters after the training is completed.

[0148] The electronic device in the embodiment of the present application can be a user terminal device, a server, other computing devices, or a cloud server. Figure 12 This is a hardware structure diagram of an electronic device according to an embodiment of the present application. The electronic device may include a processor 601 and a memory 602 storing computer program instructions. When the processor 601 executes the computer program instructions, the process or function of any of the above-mentioned embodiments is implemented.

[0149] Specifically, the processor 601 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application. The memory 602 may include a large-capacity memory for data or instructions. For example, the memory 602 may be at least one of the following: a hard disk drive (HDD), a read-only memory (ROM), a random access memory (RAM), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, a universal serial bus (USB) drive, or other physical / tangible memory storage device. For another example, the memory 602 may include a removable or non-removable (or fixed) medium. For another example, the memory 602 may be inside or outside the integrated gateway disaster recovery device. The memory 602 may be a non-volatile solid-state memory. In other words, the memory 602 typically includes a tangible (non-transitory) computer-readable storage medium (such as a memory device) encoded with computer-executable instructions, and when the software is executed (such as by one or more processors), the operations described in the method of the embodiments of the present application may be performed. The processor 601 implements the process or function of any method in the above embodiments by reading and executing computer program instructions stored in the memory 602.

[0150] In one example, Figure 12The electronic device shown may also include a communication interface 603 and a bus 610. The processor 601, memory 602, and communication interface 603 are connected via the bus 610 and communicate with each other. The communication interface 603 is primarily used to implement communication between the various modules, devices, units, and / or equipment in the embodiments of the present application. The bus 610 includes hardware, software, or both, and can couple the components of the online data traffic metering device to each other. For example, the bus may include at least one of the following: an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industrial Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Area Bus (VLB) bus, or other suitable buses. The bus 610 may include one or more buses. Although the embodiments of the present application describe or illustrate a specific bus, the embodiments of the present application may consider any suitable bus or interconnection method.

[0151] In combination with the method in the above embodiments, an embodiment of the present application also provides a computer-readable storage medium, which stores computer program instructions. When the computer program instructions are executed by a processor, they implement the process or function of any method in the above embodiments.

[0152] In addition, an embodiment of the present application further provides a computer program product, which stores computer program instructions. When the computer program instructions are executed by a processor, the process or function of any one of the methods in the above embodiments is implemented.

[0153] The flowcharts and / or block diagrams of the methods, devices, systems and computer program products of the embodiments of the present application are described above by way of example, and various aspects thereof are described. It should be understood that each box in the flowchart and / or block diagram or a combination thereof may be implemented by computer program instructions, or may be implemented by dedicated hardware that performs a specified function or action, or may be implemented by a combination of dedicated hardware and computer instructions. For example, these computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to form a machine that enables these instructions executed by such a processor to enable the implementation of the functions / actions specified in each box in the flowchart and / or block diagram or a combination thereof. Such a processor may be a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit.

[0154] The functional blocks shown in the block diagram of the embodiment of the present application can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc.; when implemented in software, it is a program or code segment used to perform the required task. The program or code segment can be stored in a memory or transmitted on a transmission medium or a communication link via a data signal carried in a carrier. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0155] It should be noted that the present application is not limited to the specific configurations and processes described above or shown in the figures. The above is only a specific implementation method of the present application. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the described system, device, module or unit can refer to the corresponding process in the method embodiment without further description. It should be understood that the scope of protection of the present application is not limited to this. Any technician familiar with the technical field can think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the scope of protection of the present application.

Claims

1. A method for generating a career planning model, characterized in that: include: Acquire a first training data set, the first training data set including occupational research information and desensitized resume information and position information, wherein the occupational research information includes information on the correspondence between occupations and functions; Using the first training data set, the general large model is pre-trained using the autoregressive method to obtain a large model for the recruitment field; Generate an instruction fine-tuning training data set based on multiple job seeker information and expert career planning suggestions corresponding to each job seeker information; Using the instruction fine-tuning training data set, the recruitment domain big model is trained with instruction fine-tuning to obtain the career planning big model; Inputting the desensitized information of multiple target job seekers into the career planning model to obtain multiple groups of expert career planning information output by the career planning model, each group of expert career planning information including multiple randomly ordered expert career planning suggestions; The career planning information of each group of experts is annotated, and the multiple career planning suggestions in the annotated career planning information of each group of experts are ranked according to their merits; Based on the information of the plurality of target job seekers and the annotated expert career planning information, forming a second training data set; According to the ranking of merit, the training data in the second training data set are sequentially input into the general large model, and a reward model is obtained after the training is completed, wherein the loss function used in the training is based on the difference between the output value of the reward model corresponding to the first input data and the output value of the reward model corresponding to the later input data; The career planning model includes a career planning model with fixed parameters and a career planning model with fine-tuned parameters, and a first expectation function is constructed based on the reward model and the career planning model with fixed parameters; Based on the target job seeker information and the first expected function, the career planning model with fine-tuned parameters is trained, and after the training is completed, a first version of the career planning model with updated parameters is obtained.

2. The generation method according to claim 1, characterized in that: Each of the plurality of job seeker information includes at least one personal information of the job seeker; The generating of instruction fine-tuning training data set based on multiple job seeker information and expert career planning suggestions corresponding to each job seeker information further includes: At least one piece of personal information in each job applicant's information is concatenated into a long text; Embed the long text and the corresponding expert career planning suggestions into the corresponding generation template, wherein the generation template includes an instruction part and an output value part, the instruction part is used to write the long text, and the output value part is used to write the corresponding expert career planning suggestions; An instruction fine-tuning training data set required for instruction fine-tuning is generated through the generation template.

3. The career planning generation method according to claim 2, characterized in that: The at least one personal information of the job seeker includes at least one of the following: the job seeker's skill status information, the job seeker's living status information, the job seeker's personality assessment information, and the job seeker's future expectation information.

4. The generation method according to claim 1, characterized in that: The loss function is: Among them, y w and l Both represent career planning, w is sorted in y l Previous career planning, x is the prompt word from the second training data set, r θ (x,y w ) is the output value of the reward model corresponding to the first input data, r θ (x,y l ) is the output value of the reward model corresponding to the input data, σ is the coefficient, and K is the number of career planning suggestions corresponding to the same target job seeker information.

5. The generation method according to claim 1, characterized in that: The parameter-fine-tuning career planning big model is trained based on the target job seeker information and the first expected function, including: scoring the career planning output by the parameter-fine-tuning career planning big model based on the reward model, and constraining the parameter-fine-tuning career planning big model based on the parameter-fixed career planning big model to determine the degree of deviation of the parameters.

6. The generation method according to claim 5, characterized in that: The first expectation function is: Among them, x RL is the prompt word from the training data of the career planning model with fine-tuned training parameters, y is the output word, RM represents the reward model, and LLM SFT Represents a large model of career planning with fixed parameters; is the first version of the career planning model, parameterized by φ, β is the coefficient, Represents the KL divergence loss of the career planning model with fixed parameters and the first version of the career planning model.

7. The generation method according to claim 6, characterized in that: Also includes: Based on the perplexity function, a constraint function is added to the first expectation function to obtain a second expectation function, and based on the second expectation function, the career planning model with fine-tuned parameters is trained, and after the training is completed, a second version of the career planning model with updated parameters is obtained; wherein, The constraint function is: The second expected function is: Where x is the prompt word from the training data of the career planning model with fine-tuned training parameters, and x pretrain is the prompt word from the first training data set, RM(x RL , y) is the preference loss of the reward model, and the perplexity function is negative It is the KL divergence loss between the career planning model with fixed parameters and the career planning model with fine-tuned parameters.

8. A device for generating a career planning model, characterized in that: include: An acquisition module, used to acquire a first training data set, wherein the first training data set includes occupational research information and desensitized resume information and position information, wherein the occupational research information includes information on the correspondence between occupations and functions; A pre-training module, used to use the first training data set and an autoregressive method to pre-train the general large model to obtain a large model for the recruitment field; A generation module, for generating an instruction fine-tuning training data set based on a plurality of job seeker information and an expert career planning suggestion corresponding to each job seeker information; An instruction fine-tuning training module is used to perform instruction fine-tuning training on the recruitment domain big model using an instruction fine-tuning training data set to obtain a career planning big model; The first input module is used to input the desensitized information of multiple target job seekers into the career planning model to obtain multiple groups of expert career planning information output by the career planning model, each group of expert career planning information includes multiple randomly ordered expert career planning suggestions; A labeling module is used to label the career planning information of each group of experts, and the career planning suggestions in the labeled career planning information of each group of experts are sorted according to their merits; A second training data set forming module, configured to form a second training data set based on the plurality of target job seekers' information and the annotated expert career planning information; A second input module, according to the merit ranking, sequentially inputs the training data in the second training data set into the general large model, and obtains a reward model after the training is completed, wherein the loss function used in the training is based on the difference between the output value of the reward model corresponding to the first input data and the output value of the reward model corresponding to the later input data; A construction module, wherein the career planning model includes a career planning model with fixed parameters and a career planning model with fine-tuned parameters, and the construction module is used to construct a first expectation function based on the reward model and the career planning model with fixed parameters; A training module is used to train the career planning model with fine-tuned parameters based on the target job seeker information and the first expected function, and obtain a first version of the career planning model with updated parameters after the training is completed.

9. An electronic device, characterized in that: The electronic device comprises: a processor and a memory storing computer program instructions; when the electronic device executes the computer program instructions, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product, characterized in that It comprises computer program instructions, which, when executed by a processor, implement the method according to any one of claims 1 to 7.

Images