Figure portrait construction method and system based on language model and reinforcement learning

Abstract

The invention belongs to the technical field of power system personnel management and intelligent evaluation, and discloses a language model and reinforcement learning-based figure portrait construction method and system, which combines a pre-training language model (such as RoBERTa) and a reinforcement learning algorithm to perform feature analysis on multi-dimensional skills of trainees so as to construct a skill figure portrait. The problems that in the prior art, skill figure portrait construction is single in dimension and lacks dynamicity, and unstructured data cannot be fully utilized are solved. The method is used for carrying out personalized training, intelligent post distribution and risk early warning, and improving the operation safety of a power grid and the personnel scheduling efficiency.

Description

Technical Field

[0001] This invention belongs to the field of power system personnel management and intelligent assessment technology, specifically involving a method and system for constructing user profiles based on language models and reinforcement learning. Background Technology

[0002] The safe and stable operation of the power grid depends not only on the reliability of equipment but also, and more importantly, on the practical skills and emergency response capabilities of the operators. For a long time, trainee skills assessments have relied primarily on periodic examinations, manual evaluations, and the accumulation of daily experience. While these methods can reflect basic personnel qualities to some extent, they often remain at a static and singular evaluation level, failing to reveal their true capabilities in complex tasks and emergencies. Furthermore, such assessments typically fail to create a unified competency profile, lacking a systematic characterization of personnel's comprehensive qualities, behavioral patterns, and learning potential.

[0003] As the power grid continues to expand, the operating environment becomes increasingly complex, leading to a surge in unstructured data such as dispatch logs, accident reports, and emergency drill records. This data contains a wealth of details reflecting personnel's actual operational skills and thought processes, but traditional methods struggle to utilize it effectively, resulting in the neglect of significant potential value. Simultaneously, skills assessment results are often updated only within the assessment cycle, lacking dynamism and failing to reflect timely trends in personnel capabilities. This limits the accurate formulation of job assignments and training plans. Current profiling technologies in the power industry remain relatively rudimentary, typically constructing fixed profiles based solely on static information, failing to reflect the real-time evolution of personnel capabilities and the correlation of multi-dimensional characteristics.

[0004] In recent years, deep learning and natural language processing technologies have developed rapidly, especially pre-trained language models based on the Transformer architecture (such as BERT and RoBERTa), which have shown outstanding performance in semantic understanding and knowledge extraction, capable of extracting skill-related information from large amounts of unstructured text. Meanwhile, reinforcement learning has been widely applied in dynamic optimization and decision recommendation, continuously adjusting skill weights based on task performance feedback to achieve adaptive evaluation and optimization. This provides a technological foundation for building continuously updated dynamic user profiles.

[0005] However, existing methods for profiling power personnel still have the following shortcomings: (1) the profile dimensions are limited and multi-source heterogeneous data are not fully integrated; (2) the profile construction is mostly based on static labels and lacks understanding of behavioral semantics and implicit skills; (3) there is a lack of a self-learning mechanism based on task feedback, making it difficult to achieve continuous optimization and intelligent evolution of the profile. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for constructing skill profiles based on language models and reinforcement learning. By combining pre-trained language models (such as RoBERTa) with reinforcement learning algorithms, multi-dimensional skill features of trainees are analyzed to construct skill profiles. This addresses the problems of existing skill profile construction methods, such as limited dimensionality, lack of dynamism, and inability to fully utilize unstructured data. The results are applicable to personalized training, intelligent job allocation, and risk warning, thereby improving the safety of power grid operation and the efficiency of personnel scheduling.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] Firstly, a method for constructing user profiles based on language models and reinforcement learning is provided, including:

[0009] Collect and preprocess historical multi-source data related to the skill level of individuals, including structured and unstructured text data.

[0010] A language model is used to output unstructured semantic vectors from the preprocessed unstructured text data. The unstructured semantic vectors are then fused with the most similar coarse-grained skill prototype vectors and fine-grained skill prototype vectors in the skill prototype library to obtain the skill semantic representation vector.

[0011] The skill semantic representation vector is weighted and fused with the preprocessed structured text data to obtain the character skill representation vector;

[0012] Collect real-time running information of characters completing the latest tasks, use the real-time running information and character skill representation vector as the state of the reinforcement learning algorithm, generate rewards based on the current state, and use the rewards to update the skill weight vector;

[0013] The skill weight vector is used to weight the character skill representation vector to obtain the skill score vector, and a multi-dimensional character profile is constructed based on the skill score vector.

[0014] Several alternative methods are provided below, but they are not intended as additional limitations on the overall solution above. They are merely further additions or optimizations. Provided there are no technical or logical contradictions, each alternative method can be combined individually with respect to the overall solution above, or multiple alternative methods can be combined with each other.

[0015] Preferably, the preprocessing includes:

[0016] Perform missing value imputation, outlier detection, and normalization operations on structured text data;

[0017] Perform word segmentation, stop word removal, and vectorization on unstructured text data;

[0018] The structured and unstructured text data after cleaning are aligned in dimension or scaled using a mapping function, and the structured and unstructured text features are output.

[0019] Preferably, the skill prototype library is constructed as follows:

[0020] Skill levels are divided into coarse-grained and fine-grained levels, with each fine-grained level belonging to a coarse-grained level. The coarse-grained level is the task capability level, and the fine-grained level is the operation action level.

[0021] Descriptive statements reflecting each skill level are extracted from relevant skill level materials. After cleaning and standardizing the descriptive statements, they are input into a language model to obtain coarse-grained skill prototype vectors and fine-grained skill prototype vectors.

[0022] Preferably, the process of fusing the unstructured semantic vector with the most similar coarse-grained and fine-grained skill prototype vectors in the skill prototype library to obtain the skill semantic representation vector includes:

[0023] By using similarity measurement, we obtain the coarse-grained skill prototype vector that is most similar to the unstructured semantic vector, and denote it as the coarse-grained skill activation vector.

[0024] By using similarity measurement, we obtain the fine-grained skill prototype vector that is most similar to the unstructured semantic vector, and denote it as the fine-grained skill activation vector.

[0025] The coarse-grained skill activation vector and the fine-grained skill activation vector are projected onto a unified skill dimension space through a linear mapping layer and then fused to obtain a skill semantic representation vector.

[0026] Preferably, the step of generating a reward based on the current state includes:

[0027] A task success flag is generated based on the task completion status. If the task completion status is successful, the task success flag is 1; otherwise, the task success flag is 0.

[0028] A task operation flag is generated based on the task operation status. If the task operation status indicates that there is an error, the task operation flag is 1; otherwise, the task operation flag is 0.

[0029] Convert the time required to complete the task into a count value in seconds, and record it as the task time stamp;

[0030] The reward is obtained by weighted summing of the negative values ​​of the task success marker, the task operation marker, and the task time marker.

[0031] Preferably, the method of updating the skill weight vector using rewards includes:

[0032] The reward is weighted using the learning rate and then added to the skill weight vector before the update to obtain the updated skill weight vector.

[0033] As a preferred option, job matching based on multi-dimensional person profiles is also included, including:

[0034] Obtain descriptive statements of job matching requirements, clean and standardize the descriptive statements, and then input them into a language model to obtain job matching feature vectors;

[0035] Calculate the semantic ability matching degree between the person's skill representation vector and the job matching feature vector;

[0036] The semantic ability matching degree is then weighted with the comprehensive score of the skill score vector in the multi-dimensional character profile to obtain the final matching degree.

[0037] The skill score vector for each dimension of the multi-dimensional persona is greater than the threshold of the single skill requirement for the job, and the job with the highest final match degree is selected.

[0038] Secondly, a character profiling system based on language models and reinforcement learning is provided, including:

[0039] The data acquisition module is used to collect and preprocess historical multi-source data related to the skill level of individuals. The historical multi-source data includes structured text data and unstructured text data.

[0040] The data representation module is used to output unstructured semantic vectors from preprocessed unstructured text data using a language model, and to fuse the unstructured semantic vectors with the most similar coarse-grained skill prototype vectors and fine-grained skill prototype vectors in the skill prototype library to obtain skill semantic representation vectors; and to perform weighted fusion of skill semantic representation vectors with preprocessed structured text data to obtain character skill representation vectors.

[0041] The real-time update module is used to collect real-time running information of the character completing the latest task. The real-time running information and the character's skill representation vector are used as the state of the reinforcement learning algorithm. Rewards are generated based on the current state, and the skill weight vector is updated using the rewards.

[0042] The task profile construction module is used to weight the character skill representation vector using the skill weight vector to obtain the skill score vector, and then construct a multi-dimensional character profile based on the skill score vector.

[0043] The method and system for constructing user profiles based on language models and reinforcement learning provided by this invention have the following advantages compared with existing technologies:

[0044] 1. By employing language models to perform deep semantic modeling on unstructured data (including operation logs, accident reports, emergency response records, etc.), it is possible to extract the skill features of individuals more accurately, overcoming the limitations of traditional methods in terms of incomplete semantic understanding and insufficient representation capabilities.

[0045] 2. Multi-granularity hierarchical semantic modeling: Based on the power grid job competency model, a multi-granularity skill system combining coarse-grained task capabilities and fine-grained operational skills is constructed. A skill prototype vector library is introduced, and an interpretable skill semantic representation is built through a similarity activation mechanism between coarse-grained and fine-grained capabilities. This mechanism enables the system to clearly identify the capability category corresponding to a textual behavior and the activation degree of its internal skill items, achieving a precise mapping from behavioral semantics to capability items. This not only improves the professionalism of the personnel skill profile but also enhances the interpretability of the model results, facilitating its application in business scenarios such as power grid training, assessment, and safety management.

[0046] 3. By combining reinforcement learning mechanisms, skill weights are adaptively updated based on feedback information (such as success rate, time consumption, and error type) after task execution, realizing the dynamic evolution and continuous optimization of character skill profiles, rather than remaining at the static evaluation level.

[0047] 4. This invention can integrate multi-source heterogeneous data to form a multi-dimensional and dynamic profile of the user, providing a scientific basis for power grid task allocation, risk warning and precision training, and effectively improving the safety, reliability and intelligence level of power grid operation. Attached Figure Description

[0048] Figure 1 This is a flowchart of the character profiling construction method based on language model and reinforcement learning according to the present invention;

[0049] Figure 2 This is a flowchart of the multi-source data acquisition and preprocessing process of the present invention;

[0050] Figure 3 This is a schematic diagram of the multi-granularity hierarchical semantic modeling process of the present invention;

[0051] Figure 4 This is a schematic diagram of the skill dynamic optimization process based on reinforcement learning in this invention;

[0052] Figure 5 This is a flowchart illustrating the process of generating a portrait in this invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention.

[0055] Example 1:

[0056] This embodiment proposes a method for constructing person profiles that combines RoBERTa's deep semantic modeling capabilities with a reinforcement learning dynamic optimization mechanism. It can automatically extract skill semantic features based on multi-source data fusion and continuously update the feature weights of person profiles through reinforcement learning. This enables dynamic modeling of personnel capabilities, job suitability, and risk levels, thereby supporting personalized training, intelligent scheduling, and safety decision-making, and comprehensively improving the safety and management efficiency of power grid operation.

[0057] like Figure 1 As shown in the figure, this embodiment provides a method for constructing a person profile based on language models and reinforcement learning, which specifically includes the following steps:

[0058] Step 1: Collect and preprocess historical multi-source data related to personnel skill levels. This historical multi-source data includes structured text data (such as exam scores, training participation data, and equipment operation records) and unstructured text data (such as operation logs, accident reports, emergency response records, and work order texts). Specific steps are as follows: Figure 2 As shown.

[0059] Step 1-1: Structured Data Acquisition and Cleaning: Structured data mainly comes from training exam scores, equipment operation logs, etc., and often contains missing values, noise, or inconsistent formats. Therefore, it needs to be cleaned through missing value imputation, outlier detection, and normalization to obtain a structured feature representation that can be used for modeling. For structured input text... , For a structured text dataset, its standardization process can be expressed as formula (1), where, For structured input text The Middle The normalized value of the dimensional feature. For structured input text The Middle The values ​​of the dimensional features before normalization and The first The mean and standard deviation of the dimensional features.

[0060] (1)

[0061] Steps 1-2: Unstructured Text Data Processing: Unstructured text data includes semantic information such as operation logs, accident reports, emergency response records, and work order texts. To achieve model understandability, word segmentation, stop word removal, and vectorization are required to convert the text data into a vector sequence that can be input into the semantic model. Given an unstructured text sequence... Its word embedding is represented by formula (2), where For words Embedded vector, Unstructured text sequence The Middle One word, Unstructured text sequence The total number of words in the text.

[0062] (2)

[0063] Steps 1-3: Multi-source feature alignment and unified input preparation: Due to the structured features after cleaning and standardization... Embedding with unstructured text sequences Since there are differences in dimension, scale and representation, we further align the features of the multi-source data and prepare a unified input, project it into a compatible representation space, and use it as a unified input for the subsequent semantic modeling module.

[0064] Through mapping function and The two types of data are dimensionally aligned or scaled to ensure that their features meet the input requirements of subsequent models. The mapping function can be implemented using common feature mapping networks, including but not limited to linear transformations, feedforward neural networks (MLPs), and attention fusion layers, depending on the specific needs. The processed data is then stored in the input buffer layer as direct input to the RoBERTa semantic modeling module.

[0065] Step 2: Using a language model (RoBERTa model in this embodiment), output unstructured semantic vectors from the preprocessed unstructured text data. Then, fuse these unstructured semantic vectors with the most similar coarse-grained and fine-grained skill prototype vectors in the skill prototype library to obtain a skill semantic representation vector. Finally, perform a weighted fusion of the skill semantic representation vector with the preprocessed structured text data to obtain an interpretable character skill representation vector. Specific steps are as follows: Figure 3 As shown.

[0066] Step 2-1, Text Semantic Encoding: Embedding the input unstructured text sequence The RoBERTa model is used for deep semantic modeling to obtain the contextual semantic representation of the text. Unstructured text sequence embedding. After RoBERTa encoding, the hidden state sequence is obtained as shown in Equation (3). Then, the unstructured semantic vector is obtained through pooling as shown in Equation (4). By pooling the semantic sequence, a global semantic vector reflecting the overall meaning of the text is obtained, which provides a foundation for subsequent skill semantic modeling.

[0067] (3)

[0068] (4)

[0069] In the formula, After encoding with RoBERTa, the hidden state sequence is obtained. For the hidden state sequence, the th A hidden state, For pooling operations, It is an unstructured semantic vector.

[0070] Step 2-2, Multi-granularity semantic layering and construction of skill prototype vector library: In order to enhance the interpretability of skill representation, this embodiment constructs a multi-granularity skill system based on the power grid job competency model, divides skills into coarse-grained task competency categories and fine-grained specific operational competencies, and constructs coarse-grained and fine-grained skill prototype vector libraries to enhance the interpretability of skill semantic expression.

[0071] Specifically, based on the job competency model and skill system of the power grid industry, skills are divided into a coarse-grained task-capability level (such as first-level skills like emergency response capability, equipment operation capability, and procedure execution capability) and a fine-grained operational action level (such as second-level skills like disconnector / disconnector operation, disconnector maintenance, accident isolation operation, switch engagement / disengagement operation, and safety procedure implementation), forming a hierarchical skill system. The coarse-grained level in this hierarchical structure includes one or more fine-grained levels. For example, emergency response capability includes accident isolation operation, sudden accident classification and judgment, and emergency shutdown / power outage operation; equipment operation capability includes disconnector / disconnector operation, core equipment start-up and shutdown operation, and disconnector maintenance; and procedure execution capability includes safety procedure implementation, equipment operation procedure execution, and maintenance procedure execution.

[0072] Based on this, core keywords are predefined for each skill item. For example, the core keywords for accident isolation operation include accident isolation, disconnecting the fault source, safe area, warning signs, isolation boundary, and fault isolation procedure. The core keywords for knife switch / disconnector switch operation include knife switch, disconnector switch, operation sequence, pre-operation inspection, post-operation confirmation, and prevention of misoperation. It is easy to understand that the core keywords can be adjusted according to actual needs. Then, using document retrieval tools, descriptive statements reflecting the core requirements of each skill item are extracted from regulations, operating procedures, and training materials. After cleaning and standardization, these statements are input into a domain-fine-tuned RoBERTa model, generating coarse-grained and fine-grained skill prototype vectors, which are then combined into a skill prototype library. The hierarchical structure of the skill system constrains the subordinate relationships between coarse-grained and fine-grained skill items, guiding the aggregation and inheritance of multi-granularity skills.

[0073] Steps 2-3, Activation Vectors and Unified Mapping: After constructing the skill prototype library, calculate the unstructured semantic vectors respectively. The similarity (e.g., cosine similarity) between the skill prototype vectors and the unstructured semantic vectors is used to obtain the coarse-grained skill prototype vector, denoted as the coarse-grained skill activation vector; the fine-grained skill prototype vector is also obtained, denoted as the fine-grained skill activation vector. Subsequently, the above multi-granularity skill semantic information is fused, specifically by projecting it onto a unified skill dimension space through a linear mapping layer to obtain the skill semantic representation vector. The mapping process is shown in formulas (5), (6) and (7).

[0074] (5)

[0075] (6)

[0076] (7)

[0077] In the formula, Representing unstructured semantic vectors With the Coarse-grained skill prototype vector Similarity between them Representing unstructured semantic vectors With the Fine-grained skill prototype vector Similarity between them For coarse-grained skill activation vectors, For fine-grained skill activation vectors, This is a coarse-grained skill mapping matrix. It is a fine-grained skill mapping matrix, which is pre-defined according to the hierarchical structure of the skill system.

[0078] Steps 2-4: Encode the structured features With skill semantic representation vector Integration to form a character skill representation vector As shown in formula (8), where, This is an adjustable parameter that determines the weight of features from different sources in the final representation.

[0079] (8)

[0080] Step 3: Collect real-time operational information on the completion of the latest task by the user. Use this real-time operational information and the user's skill representation vector as the state of the reinforcement learning algorithm. Generate a reward based on the current state and update the skill weight vector using the reward. This embodiment is based on dynamic optimization of reinforcement learning; that is, during the actual operation of the system, the system dynamically adjusts the weights of each skill according to the user's performance, thereby achieving continuous updating of the ability profile. Specific steps are as follows... Figure 4 As shown.

[0081] Step 3-1, Running Status Construction: In each task cycle After the task is completed, the system collects task operation information, including indicators such as task completion status, task operation status, execution time, execution efficiency, and operation quality, and combines this information with the character's skill representation vector. Build the current running state Current running status This is used to generate evaluation signals that reflect the quality of the execution, in order to measure the real-time performance of personnel across different skill dimensions.

[0082] Step 3-2, Reward Signal Generation: Based on the current operating status Generate reward signals For example, if the task is successful and takes little time, the reward value is higher; if an error occurs, the reward value is lower or even negative. In this embodiment, the reward function can be defined as formula (9), which is the weighted sum of the negative values ​​of the task success marker, the task operation marker, and the task time marker to obtain the reward. The weighting coefficients are as follows: Success is a task success flag generated based on the task completion status. If the task completion status is successful, the task success flag is 1; otherwise, the task success flag is 0. Error is a task operation flag generated based on the task operation status. If the task operation status indicates that there is an error, the task operation flag is 1; otherwise, the task operation flag is 0. Time represents the task time flag, which converts the time required to complete the task into a count value in seconds.

[0083] (9)

[0084] Step 3-3, Online Update of Skill Weights: This embodiment employs reinforcement learning, combining the aforementioned evaluation signals with skill representation vectors to update the skill weight vectors online. This allows the system to adaptively adjust skill weights based on the actual performance of personnel in different tasks. (Predefined) This is the skill weight vector at the current moment (a multi-dimensional vector, meaning one skill corresponds to one weight). The skill weights are adaptively adjusted based on performance. The update rule is as shown in formula (10). For the first The current weight of each skill (initial value is preset). Corresponding to real-time reward signals, The learning rate is used. The latest skill weight vector is used in the weighted calculation of the generated persona, enabling the final persona to reflect the changing trends of an individual's abilities across various skill dimensions in real time. As the system continuously receives new task execution results, the skill weight vector will be continuously adjusted, transforming the persona from a static representation to a dynamically evolving depiction of abilities.

[0085] (10)

[0086] Step 4: Weight the character's skill representation vector using the skill weight vector to obtain the skill rating vector, and construct a multi-dimensional character profile based on the skill rating vector. Specific steps are as follows: Figure 5 As shown.

[0087] Step 4-1, Dynamic Skill Scoring: First, the system will use the character skill representation vector obtained in Step 2 to... Compared with the skill weight vector optimized online in step 3 The data is then integrated to generate a skill score vector that reflects the current ability status of the personnel. The skill scoring vector incorporates semantic information from both structured data and unstructured text, as well as dynamic adjustment factors driven by actual task performance, enabling a comprehensive characterization of an individual's ability level across different skill dimensions.

[0088] Step 4-2: Multi-dimensional Personnel Profile Construction: A multi-dimensional personnel profile is constructed based on the skill scoring vector, including multiple capability dimensions such as operational ability, procedure execution, emergency response, and risk control. Taking a scoring range of 0-10 as an example, Person 1: Operational ability 9.5, Procedure execution 9.2, Emergency response 8.8, and Risk control 9.1; Person 2: Operational ability 7.5, Procedure execution 8.2, Emergency response 6.8, and Risk control 7.1; Person 3: Operational ability 5.5, Procedure execution 6.2, Emergency response 4.5, and Risk control 6.8. It should be noted that the dimensions in the personnel profile can be adjusted as needed. For example, it can also include skill mastery, equipment operation proficiency, risk control ability, execution efficiency and operational stability, or it can correspond to a coarse-grained level.

[0089] Personnel profiles can be visualized using radar charts, trend graphs, or skill matrices to reveal an individual's strengths, weaknesses, and trends in skill development. Because the skill scoring vector integrates static skill representations with dynamic weights, the generated profile is updated in real-time and evolves continuously with task performance.

[0090] The person profile constructed in this embodiment can be used for practical applications such as task / job allocation (matching the best candidate), personalized training recommendations, performance appraisal and promotion evaluation.

[0091] In one embodiment, taking task / job allocation as an example, a task / job allocation method based on a person profile is provided. This includes obtaining descriptive statements of job matching requirements; cleaning and standardizing the descriptive statements before inputting them into a language model to obtain a job matching feature vector; calculating the semantic ability matching degree between the person's skill representation vector and the job matching feature vector; and weighting the semantic ability matching degree with the skill rating vector in the multi-dimensional person profile to obtain the final matching degree. The goal is to match the job with the highest skill score vector in each dimension of the multi-dimensional profile, where the skill score vector in each dimension is greater than the threshold for the individual skill requirement of the job.

[0092] In one embodiment, taking risk identification as an example, a task execution risk early warning method based on a person's profile is provided. This method calculates the matching degree between a person and a specific task or position, combining task requirements or job competency requirements. This matching degree comprehensively considers the person's semantic abilities (based on skill representation) and recent actual performance (based on dynamic skill scoring). Risk identification is performed based on low scores in key competency dimensions and overall matching performance. Early warnings are triggered when competency is insufficient or the matching is too low, providing support for task decision-making and safety management. Specifically, this involves targeting the semantic vector of task requirements. Calculating the target matching degree includes semantic capability matching (based on...) ) and dynamic performance matching (based on The final matching degree is obtained as shown in formula (11).

[0093] (11)

[0094] In the formula, Character skill representation vector Job-matching feature vector The final match degree between them Character skill representation vector Job-matching feature vector The semantic matching degree between them The overall score for the skill rating vector in the multidimensional character profile. For similarity functions, such as cosine similarity, For comprehensive scoring functions, This is the weighted ratio.

[0095] Based on skill score vector and final match Assess potential risks when a score on a certain skill dimension falls below a risk threshold. or overall matching degree Below the threshold The risk warning is triggered as shown in formula (12).

[0096] (12)

[0097] Example 2:

[0098] This embodiment provides a character profile construction system based on language models and reinforcement learning, including a data acquisition module, a data representation module, a real-time update module, and a task profile construction module, wherein:

[0099] The data acquisition module is used to collect and preprocess historical multi-source data related to the skill level of individuals. The historical multi-source data includes structured text data and unstructured text data.

[0100] The data representation module is used to output unstructured semantic vectors from preprocessed unstructured text data using a language model, and to fuse the unstructured semantic vectors with the most similar coarse-grained skill prototype vectors and fine-grained skill prototype vectors in the skill prototype library to obtain skill semantic representation vectors; and to perform weighted fusion of skill semantic representation vectors with preprocessed structured text data to obtain character skill representation vectors.

[0101] The real-time update module is used to collect real-time running information of the character completing the latest task. The real-time running information and the character's skill representation vector are used as the state of the reinforcement learning algorithm. Rewards are generated based on the current state, and the skill weight vector is updated using the rewards.

[0102] The task profile construction module is used to weight the character skill representation vector using the skill weight vector to obtain the skill score vector, and then construct a multi-dimensional character profile based on the skill score vector.

[0103] For specific limitations on the character profiling system based on language models and reinforcement learning, please refer to the limitations on the character profiling method based on language models and reinforcement learning mentioned above, which will not be repeated here.

[0104] Example 3:

[0105] This embodiment provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the steps of the character profile construction method based on language model and reinforcement learning in Embodiment 1.

[0106] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0107] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0108] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A method for constructing a persona based on language models and reinforcement learning, characterized in that, The method for constructing user profiles based on language models and reinforcement learning includes: Collect and preprocess historical multi-source data related to the skill level of individuals, including structured and unstructured text data. A language model is used to output unstructured semantic vectors from the preprocessed unstructured text data. The unstructured semantic vectors are then fused with the most similar coarse-grained skill prototype vectors and fine-grained skill prototype vectors in the skill prototype library to obtain the skill semantic representation vector. The skill semantic representation vector is weighted and fused with the preprocessed structured text data to obtain the character skill representation vector; Collect real-time running information of characters completing the latest tasks, use the real-time running information and character skill representation vector as the state of the reinforcement learning algorithm, generate rewards based on the current state, and use the rewards to update the skill weight vector; The skill weight vector is used to weight the character skill representation vector to obtain the skill score vector, and a multi-dimensional character profile is constructed based on the skill score vector.

2. The method for constructing a persona based on language models and reinforcement learning according to claim 1, characterized in that, The preprocessing includes: Perform missing value imputation, outlier detection, and normalization operations on structured text data; Perform word segmentation, stop word removal, and vectorization on unstructured text data; The structured and unstructured text data after cleaning are aligned in dimension or scaled using a mapping function, and the structured and unstructured text features are output.

3. The method for constructing a persona based on language models and reinforcement learning according to claim 1, characterized in that, The skill prototype library is constructed as follows: Skill levels are divided into coarse-grained and fine-grained levels, with each fine-grained level belonging to a coarse-grained level. The coarse-grained level is the task capability level, and the fine-grained level is the operation action level. Descriptive statements reflecting each skill level are extracted from relevant skill level materials. After cleaning and standardizing the descriptive statements, they are input into a language model to obtain coarse-grained skill prototype vectors and fine-grained skill prototype vectors.

4. The method for constructing a persona based on language models and reinforcement learning according to claim 1, characterized in that, The process of fusing unstructured semantic vectors with the most similar coarse-grained and fine-grained skill prototype vectors in the skill prototype library to obtain a skill semantic representation vector includes: By using similarity measurement, we obtain the coarse-grained skill prototype vector that is most similar to the unstructured semantic vector, and denote it as the coarse-grained skill activation vector. By using similarity measurement, we obtain the fine-grained skill prototype vector that is most similar to the unstructured semantic vector, and denote it as the fine-grained skill activation vector. The coarse-grained skill activation vector and the fine-grained skill activation vector are projected onto a unified skill dimension space through a linear mapping layer and then fused to obtain a skill semantic representation vector.

5. The method for constructing a persona based on language models and reinforcement learning according to claim 1, characterized in that, The generation of rewards based on the current state includes: A task success flag is generated based on the task completion status. If the task completion status is successful, the task success flag is 1; otherwise, the task success flag is 0. A task operation flag is generated based on the task operation status. If the task operation status indicates that there is an error, the task operation flag is 1; otherwise, the task operation flag is 0. Convert the time required to complete the task into a count value in seconds, and record it as the task time stamp; The reward is obtained by weighted summing of the negative values ​​of the task success marker, the task operation marker, and the task time marker.

6. The method for constructing a persona based on language models and reinforcement learning according to claim 1, characterized in that, The method of updating the skill weight vector using rewards includes: The reward is weighted using the learning rate and then added to the skill weight vector before the update to obtain the updated skill weight vector.

7. The method for constructing a persona based on language models and reinforcement learning according to claim 1, characterized in that, It also includes job matching based on multi-dimensional profiles, including: Obtain descriptive statements of job matching requirements, clean and standardize the descriptive statements, and then input them into a language model to obtain job matching feature vectors; Calculate the semantic ability matching degree between the person's skill representation vector and the job matching feature vector; The semantic ability matching degree is then weighted with the comprehensive score of the skill score vector in the multi-dimensional character profile to obtain the final matching degree. The skill score vector for each dimension of the multi-dimensional persona is greater than the threshold of the single skill requirement for the job, and the job with the highest final match degree is selected.

8. A character profiling construction system based on language models and reinforcement learning, characterized in that, The character profiling system based on language models and reinforcement learning includes: The data acquisition module is used to collect and preprocess historical multi-source data related to the skill level of individuals. The historical multi-source data includes structured text data and unstructured text data. The data representation module is used to output unstructured semantic vectors from preprocessed unstructured text data using a language model, and to fuse the unstructured semantic vectors with the most similar coarse-grained skill prototype vectors and fine-grained skill prototype vectors in the skill prototype library to obtain skill semantic representation vectors; and to perform weighted fusion of skill semantic representation vectors with preprocessed structured text data to obtain character skill representation vectors. The real-time update module is used to collect real-time running information of the character completing the latest task. The real-time running information and the character's skill representation vector are used as the state of the reinforcement learning algorithm. Rewards are generated based on the current state, and the skill weight vector is updated using the rewards. The task profile construction module is used to weight the character skill representation vector using the skill weight vector to obtain the skill score vector, and then construct a multi-dimensional character profile based on the skill score vector.

Images