Digital employee application method and device, equipment, storage medium and program product

Abstract

The invention provides a digital employee application method and device, equipment, a storage medium and a program product, and relates to the technical field of artificial intelligence, and the method comprises the steps: obtaining a to-be-executed business instruction of a user; inputting the to-be-executed business instruction into the digital employee model, and obtaining a response result output by the digital employee model; based on the response result, executing the to-be-executed service instruction to obtain a service execution result; wherein the digital employee model is obtained by training the initial large language model based on the sample service instruction and the sample response result corresponding to the sample service instruction. Through the above mode, the large language model and the digital employee technology are combined, so that the digital employee model has the capability of processing the natural language instruction, and can understand the complex and personalized to-be-executed business instruction of the user, thereby completing diversified business processing, and effectively improving the business execution effect.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a digital employee application method, device, equipment, storage medium and program product. Background Art

[0002] The Digital Worker is a product based on digital and artificial intelligence technologies. Implemented through Robotic Process Automation (RPA) and Artificial Intelligence (AI), it aims to break down the boundaries between humans and machines and improve work efficiency and quality. Digital Workers are used in a wide range of scenarios, including document processing, work order entry, customer service, system inspections, data collation, and financial report verification.

[0003] Existing digital employee systems or models can automate simple business processes, such as executing RPA programs in servers and automatically acquiring data. However, due to their lack of intelligence, these systems or models do not support personalized user input and struggle to execute complex business instructions, resulting in poor business execution results. Summary of the Invention

[0004] The present invention provides a digital employee application method, apparatus, device, storage medium and program product to address the defects in the prior art that the digital employee model does not support the user's personalized instruction input, has difficulty in processing the execution of complex business instructions, and leads to poor business execution results.

[0005] The present invention provides a digital employee application method, comprising: obtaining a user's business instructions to be executed; inputting the business instructions to be executed into a digital employee model to obtain a response result output by the digital employee model; and executing the business instructions to be executed based on the response result to obtain a business execution result. The digital employee model is obtained by training an initial large language model based on sample business instructions and sample response results corresponding to the sample business instructions.

[0006] According to a digital employee application method provided by the present invention, based on a response result, a pending business instruction is executed. After obtaining the business execution result, the method further includes: determining whether the business execution result meets the user's business needs; if the business execution result does not meet the user's business needs, regenerating a response result based on a digital employee model; and based on the regenerated response result, re-executing the pending business instruction and updating the business execution result until the business execution result meets the user's business needs.

[0007] According to a digital employee application method provided by the present invention, based on the response result, a pending business instruction is executed. After obtaining the business execution result, the method further includes: obtaining user feedback information; the feedback information is the user's satisfaction score for the response result; based on the feedback information, reinforcement learning training is performed on the digital employee model to optimize the digital employee model.

[0008] According to a digital employee application method provided by the present invention, reinforcement learning training is performed on a digital employee model based on feedback information to optimize the digital employee model. The method includes: fine-tuning the digital employee model through supervision to obtain a fine-tuned digital employee model; constructing a reward model; and, based on the reward model and feedback information, performing reinforcement learning training on the fine-tuned digital employee model to optimize the fine-tuned digital employee model.

[0009] According to a digital employee application method provided by the present invention, before obtaining the user's business instructions to be executed, it also includes: obtaining a basic data set; the basic data set includes work order data and product document data of the user's business; based on the basic data set, constructing multiple business instruction question and answer pairs; each business instruction question and answer pair includes a business instruction and a response result corresponding to the business instruction; inputting the multiple business instruction question and answer pairs into a pre-trained synonymous sentence conversion model, performing synonymous sentence conversion, and obtaining multiple synonymous business instruction question and answer pairs output by the synonymous sentence conversion model; each synonymous business instruction question and answer pair includes a business instruction and a response result corresponding to the business instruction; based on the multiple business instruction question and answer pairs and the multiple synonymous business instruction question and answer pairs, constructing a training sample set; the training sample set includes multiple sample business instructions and sample response results corresponding to each sample business instruction; based on the training sample set, training the initial large language model to obtain an initial digital employee model; performing LoRA fine-tuning training on the initial digital employee model to obtain a digital employee model.

[0010] According to a digital employee application method provided by the present invention, LoRA fine-tuning training is performed on an initial digital employee model to obtain a digital employee model, including: determining a target network layer of the initial digital employee model; adding trainable parameters to the target network layer based on a low-rank approximation method; and fine-tuning the target network layer based on a training sample set to obtain a digital employee model.

[0011] The present invention also provides a digital employee application device, comprising: an acquisition module for acquiring a user's business instructions to be executed; a generation module for inputting the business instructions to be executed into a digital employee model and obtaining a response result output by the digital employee model; and an execution module for executing the business instructions to be executed based on the response result and obtaining a business execution result; wherein the digital employee model is obtained by training an initial large language model based on sample business instructions and sample response results corresponding to the sample business instructions.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, any one of the above-mentioned digital employee application methods is implemented.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which implements any of the above-mentioned digital employee application methods when executed by a processor.

[0014] The present invention also provides a computer program product, including a computer program, which implements any of the above-mentioned digital employee application methods when executed by a processor.

[0015] The digital employee application method, device, equipment, storage medium and program product provided by the present invention combine the large language model and digital employee technology, and train the digital employee model based on the large language model based on sample business instructions and the sample response results corresponding to the sample business instructions. The digital employee model has the ability to process natural language instructions, can understand the user's complex and personalized business instructions to be executed, and generate response results corresponding to the business instructions to be executed, and then execute the business instructions to be executed based on the response results to obtain business execution results. Since the large language model has extremely strong generalization and versatility, the digital employee model built based on the large language model can handle the execution of complex business instructions, thereby completing diversified business processing, and can effectively improve the business execution effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 This is one of the flow charts of the digital employee application method provided by the present invention.

[0018] Figure 2 This is the second flow chart of the digital employee application method provided by the present invention.

[0019] Figure 3 It is a structural diagram of the digital employee application device provided by the present invention.

[0020] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0021] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0022] See also Figure 1 , Figure 1 This is one of the flow charts of the digital employee application method provided by the present invention. In this embodiment, the digital employee application method includes steps S110 to S130, each of which is as follows: S110: Obtain the user's business instructions to be executed.

[0023] S120: Input the business instruction to be executed into the digital employee model, and obtain a response result output by the digital employee model.

[0024] S130: Based on the response result, execute the pending business instruction and obtain the business execution result.

[0025] Among them, the digital employee model is obtained by training the initial large language model based on sample business instructions and sample response results corresponding to the sample business instructions.

[0026] Large language models are a revolutionary technology in the field of artificial intelligence, integrating the most cutting-edge achievements in natural language processing technology. They rely on the neural network architecture of deep learning, especially the Transformer framework, and achieve efficient learning and processing of language data by simulating the operation of neurons in the human brain. The core of the Transformer framework lies in the self-attention mechanism, which allows large language models to flexibly focus on different parts of the input sequence, thereby accurately capturing long-range dependencies, which is crucial for understanding the complex structure of language. With the continuous evolution of technology, large language models contain billions or even trillions of parameters. This parameter scale enables them to learn extremely subtle and deep laws in language, thereby demonstrating excellent language generation and understanding capabilities. They can not only process contextual information in long conversations, but also accurately judge user intentions and perform operations.

[0027] This embodiment applies a large language model to the field of digital employees, which can enable the digital employee model to have powerful contextual and generalization capabilities, thereby improving the business processing capabilities of the digital employee model.

[0028] To facilitate understanding of the application process of digital employees (i.e., digital employee model) in this embodiment, Figure 2 Let’s take this as an example to explain.

[0029] See also Figure 2 , Figure 2 This is the second flow chart of the digital employee application method provided by the present invention.

[0030] like Figure 2 As shown in the figure, during the reasoning and application stages, the digital employee model uses a process of repeatedly generating query words to query and generating answers based on the repeatedly generated query words until the final answer (i.e., response result) that meets the user's needs is obtained.

[0031] Assuming that the business instruction to be executed is "Close the work order numbered TK9550", the business instruction to be executed is input as user input data into the digital employee model built based on the large language model; after receiving the business instruction to be executed, the digital employee model can perform natural language understanding and processing on the business instruction to be executed, and extract user instruction information and work order number information.

[0032] Furthermore, the digital employee model will query the database based on the work order number information "TK9550" to determine whether a work order numbered TK9550 exists, thereby avoiding attempts to close non-existent work orders.

[0033] During the query process, the digital employee model will generate a series of query words and piece them together to obtain complete contextual information. The contextual information includes query instructions (such as system query instructions, user query instructions), query content (such as querying work orders or querying product documentation), query type (such as calling API queries), etc. The digital employee model can query based on the contextual information and write the query results (i.e., response results) at the end of the contextual information.

[0034] It is understandable that after obtaining the query results, the digital employee model will determine whether to continue the query: when the query results do not meet the user's business needs or the query statement is wrong, the digital employee model will continue to query until the query results meet the user's business needs.

[0035] For example, if the query instruction in the context information is wrong, the digital employee model will regenerate the query instruction and continue to query according to the regenerated query instruction until the work order numbered TK9550 is found; if the work order numbered TK9550 exists, the digital employee model can generate and display the response result "Work order TK9550 will be closed soon!" to the user.

[0036] Furthermore, after obtaining the response result, the digital employee model can execute the pending business instructions based on the response result, close the work order numbered TK9550, and obtain the business execution result.

[0037] Furthermore, after obtaining the business execution results, the digital employee model can determine whether the business execution results meet the user's business needs.

[0038] If the business execution result does not meet the user's business needs, for example, the work order numbered TK9550 fails to be closed, the digital employee model will regenerate the response result and re-execute the pending business instructions based on the re-generated response result, try to close the work order numbered TK9550 again, and update the business execution result until the business execution result meets the user's business needs.

[0039] The digital employee application method provided in this embodiment combines a large language model with digital employee technology, and trains a digital employee model constructed based on the large language model based on sample business instructions and sample response results corresponding to the sample business instructions. This enables the digital employee model to have the ability to process natural language instructions, understand the user's complex and personalized business instructions to be executed, and generate response results corresponding to the business instructions to be executed, and then execute the business instructions to be executed based on the response results to obtain business execution results. Since the large language model has extremely strong generalization and versatility, the digital employee model constructed based on the large language model can handle the execution of complex business instructions, thereby completing diversified business processing, and effectively improving business execution results.

[0040] In some embodiments, based on the response result, the business instruction to be executed is executed. After obtaining the business execution result, it also includes: judging whether the business execution result meets the business needs of the user; if the business execution result does not meet the business needs of the user, regenerating the response result based on the digital employee model; based on the regenerated response result, executing the business instruction to be executed again, updating the business execution result, until the business execution result meets the business needs of the user.

[0041] Understandably, in the early stages of digital employee model implementation, the model may not accurately generate responses that meet user business requirements, nor may it perfectly execute pending business instructions based on these responses. Therefore, after obtaining business execution results, the digital employee model must determine whether these results meet user business requirements.

[0042] If the business execution result does not meet the user's business needs, the digital employee model will regenerate the response result and, based on the regenerated response result, re-execute the pending business instructions and update the business execution result until the business execution result meets the user's business needs.

[0043] In some embodiments, based on the response result, the business instructions to be executed are executed, and after the business execution result is obtained, it also includes: obtaining user feedback information; the feedback information is the user's satisfaction score for the response result; based on the feedback information, the digital employee model is reinforced learning training to optimize the digital employee model.

[0044] Reinforcement learning, as a machine learning method, has been widely used in recent years to improve the performance and adaptability of large language models. This method can imitate the learning process of organisms in the environment and optimize the model's behavioral strategy through continuous trial and feedback. In the context of large language models, reinforcement learning can not only help the model learn how to generate natural and coherent text more effectively, but also enable it to make better decisions in specific tasks, such as providing more appropriate responses in dialogue systems or completing complex instruction tasks. By setting a suitable reward function, the large language model will receive positive or negative feedback based on the quality of its output after each interaction, thereby gradually adjusting its internal weights to achieve iterative performance improvement. For example, using a sparse reward mechanism can encourage large language models to explore and discover rare but important solutions, while dense rewards can help large language models quickly learn common and directly effective response patterns.

[0045] To effectively leverage reinforcement learning to improve the performance of large language models, researchers have adopted a variety of strategies and technical approaches. First, fine-tuning and adaptive learning are key steps. By fine-tuning the model for specific tasks or scenarios based on pre-training, and using reinforcement learning to score and adjust the model's output, large language models can improve their performance in specific domains while maintaining broad knowledge. Second, environmental interaction simulation is another important aspect. By constructing simulated environments or leveraging actual user interactions, large language models can learn through trial and error how to better predict user intent, maintain conversational coherence, or achieve designated goals. Furthermore, knowledge graph integration is also part of the reinforcement learning strategy, enabling large language models to leverage structured knowledge to enhance contextual understanding and reasoning capabilities. These capabilities are particularly important for tasks requiring precise facts or logical reasoning. Finally, algorithmic innovation and optimization: For example, incorporating the XOT algorithm and / or the Monte Carlo Tree Search (MCTS) algorithm into reinforcement learning can effectively integrate external domain knowledge, significantly enhancing the reasoning and decision-making capabilities of large language models.

[0046] In this embodiment, reinforcement learning based on human feedback (RLHF) is used to perform reinforcement learning training on the digital employee model. The core idea of ​​the reinforcement learning method based on human feedback is to use human feedback to guide the model training process, so that the model can better understand and generate data that conforms to human intentions.

[0047] Specifically, after obtaining the final business execution result, user feedback information can be obtained. The feedback information is the user's satisfaction score for the response result. Based on the feedback information, reinforcement learning training is performed on the digital employee model to optimize the digital employee model.

[0048] In some embodiments, based on feedback information, reinforcement learning training is performed on the digital employee model to optimize the digital employee model, including: supervised fine-tuning of the digital employee model to obtain a fine-tuned digital employee model; constructing a reward model; based on the reward model and feedback information, reinforcement learning training is performed on the fine-tuned digital employee model to optimize the fine-tuned digital employee model.

[0049] The training process of reinforcement learning with human feedback (RLHF) can be divided into three stages: supervised fine-tuning (SFT), reward modeling (RM), and reinforcement learning (RL).

[0050] Specifically, the digital employee model is fine-tuned through supervision to obtain a fine-tuned digital employee model.

[0051] Furthermore, in the reward modeling stage, a reward model that can simulate human preferences can be constructed.

[0052] The reward model takes a text sequence as input and outputs a reward value that aligns with human preferences. This reward value is crucial for subsequent reinforcement learning training. Building the reward model requires collecting a large amount of data. This data is typically generated by running different language models through the same text sequence and then scored by humans.

[0053] Furthermore, in the reinforcement learning (RL) stage, the fine-tuned digital employee model is subjected to reinforcement learning training based on the reward model and feedback information to optimize the fine-tuned digital employee model.

[0054] In reinforcement learning training, key concepts such as state space, action space, policy function, and value function must be defined. The action space is the set of all possible tokens, and the state space is the distribution of input sequences. The value function is derived by combining the reward model and the policy function. The policy function is determined based on the fine-tuned digital worker model trained during the supervised fine-tuning phase. By continuously adjusting the model parameters to maximize the expected reward, the model can perform better on tasks such as dialogue generation.

[0055] Furthermore, when deploying the digital employee model, a scoring mechanism can be added, allowing users to rate answers based on their satisfaction with the results. For questions with uncertainties, the model can also generate two answers simultaneously, allowing users to choose the more ideal one. Through gradual reinforcement learning adjustments, the model's output will increasingly align with user preferences.

[0056] In some embodiments, before obtaining the user's business instructions to be executed, it also includes: obtaining a basic data set; the basic data set includes the work order data and product document data of the user's business; based on the basic data set, constructing multiple business instruction question and answer pairs; each business instruction question and answer pair includes a business instruction and a response result corresponding to the business instruction; inputting multiple business instruction question and answer pairs into a pre-trained synonymous sentence conversion model, performing synonymous sentence conversion, and obtaining multiple synonymous business instruction question and answer pairs output by the synonymous sentence conversion model; each synonymous business instruction question and answer pair includes a business instruction and a response result corresponding to the business instruction; based on multiple business instruction question and answer pairs and multiple synonymous business instruction question and answer pairs, constructing a training sample set; the training sample set includes multiple sample business instructions and sample response results corresponding to each sample business instruction; based on the training sample set, training the initial large language model to obtain an initial digital employee model; performing LoRA fine-tuning training on the initial digital employee model to obtain a digital employee model.

[0057] Before obtaining the user's business instructions to be executed, a digital employee model needs to be trained.

[0058] Specifically, obtain the basic data set, which should include all work order data and product document data of daily user business.

[0059] Furthermore, the data in the basic data set is converted into the form of multiple business instruction question-answer pairs; among them, the format of the business instruction question-answer pair needs to be consistent with the format of the model inference stage, and each business instruction question-answer pair includes a business instruction and the response result corresponding to the business instruction.

[0060] Furthermore, for work order data, information such as numbers can be randomly changed; for each business instruction question and answer pair, the business instruction question and answer pair can be input into a pre-trained synonymous sentence conversion model, and the synonymous sentence conversion model can perform synonymous sentence conversion on the business instruction question and answer pair, and output a synonymous business instruction question and answer pair of the business instruction question and answer pair. It can be understood that the synonymous business instruction question and answer pair also includes a business instruction and a response result corresponding to the business instruction, and the synonymous business instruction question and answer pair has similar semantics to the business instruction question and answer pair.

[0061] Among them, the synonym conversion model is the pre-trained Paraphrase model.

[0062] By randomly changing the work order data and converting synonyms, the basic data can be amplified and enhanced to obtain more training samples.

[0063] Furthermore, a plurality of business instruction question-answer pairs and a plurality of synonymous business instruction question-answer pairs are used as a training sample set, so that the training sample set includes a plurality of sample business instructions and a sample response result corresponding to each sample business instruction.

[0064] It is understandable that in the process of constructing training samples, all useful information should be converted into the representation of question-answer pairs as much as possible; for question-answer pairs that require retrieval-enhanced generation technology, the query terms generated by the expected model should also be given when constructing the training dataset, and the query results should be simulated.

[0065] Furthermore, based on the training sample set, the initial large language model is trained to obtain the initial digital employee model; the initial digital employee model is fine-tuned using LoRA to obtain the digital employee model.

[0066] In some embodiments, LoRA fine-tuning training is performed on the initial digital employee model to obtain the digital employee model, including: determining a target network layer of the initial digital employee model; adding trainable parameters to the target network layer based on a low-rank approximation method; and fine-tuning the target network layer based on a training sample set to obtain the digital employee model.

[0067] In this embodiment, a lightweight online adaptation (Low-Rank Adaptation, LoRA) method is used to fine-tune the initial digital worker model and optimize the large model for specific instruction tasks.

[0068] LoRA fine-tuning training has many advantages: First, its lightweight nature greatly reduces computing resources and storage requirements, allowing model updates to be completed quickly under limited hardware conditions, accelerating the model iteration cycle; Second, LoRA fine-tuning training maintains the scalability of the model, that is, when new instructions or tasks are added, only the corresponding LoRA layer needs to be added without repeatedly training the entire model, which facilitates continuous learning and adaptive upgrades; In addition, LoRA fine-tuning training is efficient and flexible, and can promote the rapid deployment of models in different scenarios. Whether it is enterprise-level customized services or the specific needs of individual users, they can be met immediately and accurately.

[0069] It should be noted that the LoRA fine-tuning method uses its unique low-rank decomposition strategy to effectively adjust the pre-trained model, introducing a small number of additional parameters in the model's key layers. These additional parameters can be represented by low-rank matrices, greatly reducing the storage and computational burden. This strategy not only maintains the model's original rich knowledge reserve, but also promotes rapid improvement in its understanding and responsiveness to new instructions. The key to the LoRA fine-tuning process lies in building a lightweight adaptation layer that is compatible with the original model architecture.

[0070] Specifically, a target network layer of the initial digital employee model is determined; the target network layer is a key layer in the initial digital employee model that is sensitive to the task.

[0071] Furthermore, based on the low-rank approximation method, a small number of trainable parameters are added to the target network layer to adjust the output of the target network layer.

[0072] Among them, the newly added trainable parameters in the target network layer can interact with the original model parameters, aiming to capture the specific characteristics of the instruction task without interfering with the global knowledge structure of the model.

[0073] Furthermore, based on the training sample set, the target network layer of the initial digital employee model is fine-tuned to obtain the digital employee model.

[0074] Specifically, during the fine-tuning training process, the instruction examples and expected outputs in the training sample set are used as supervision signals, and combined with the optimization algorithm, the target network layer (also known as the LoRA layer) is gradually adjusted to ensure that the model follows the instruction intent while maintaining high quality and consistency of the output results. This method can effectively avoid the risk of overfitting of the model during the fine-tuning process and achieve efficient and accurate instruction adaptation.

[0075] The digital employee application method provided in this embodiment surpasses existing similar technical solutions in multiple dimensions, as embodied in the following three aspects: (1) Pioneering integration of big language model technology and digital employee framework: Integrating big language model technology and digital employees improves the intelligence level of digital employees. This integration enables digital employees to not only handle simple automated operations but also enable them to conduct deep learning, understand complex situations, and make reasonable decisions, breaking the limitations of traditional automation systems. The introduction of big language models enables digital employees to autonomously adapt to diverse business task environments, handle complex logic and emotional interactions, and improve business processing results.

[0076] (2) The fine-tuning of instructions for specific scenarios is another important innovation of this embodiment: through a carefully designed fine-tuning process, digital employees can accurately grasp and efficiently execute various complex instructions, whether it is work order processing, technical diagnosis or customer service, and can respond quickly and accurately. This technology breaks away from the template operation of traditional automation tools and achieves a deep understanding and personalized execution of instructions, significantly improving the flexibility and accuracy of task completion, and overcoming the shortcomings of traditional automation systems in processing complex instructions.

[0077] (3) Real-time information integration and personalized service upgrades: The addition of search-enhanced generation technology enables digital employees to access external information sources in real time and instantly update the knowledge base, ensuring the timeliness and professionalism of service information. This feature is particularly important for service scenarios that require the latest data support, such as market dynamics tracking and technical support. It enables digital employees to provide professional advice and solutions. At the same time, combined with reinforcement learning mechanisms, digital employees can continuously optimize service strategies based on user feedback, not only improving the level of service personalization, but also ensuring that service quality continues to improve over time, truly achieving dynamic synchronization and deep alignment with user needs.

[0078] The present invention also provides a digital employee application device. Figure 3 , Figure 3 3 is a schematic diagram of the structure of the digital employee application device provided by the present invention. In this embodiment, the digital employee application device includes an acquisition module 310, a generation module 320 and an execution module 330.

[0079] The acquisition module 310 is used to acquire the user's business instructions to be executed.

[0080] The generation module 320 is used to input the business instruction to be executed into the digital employee model and obtain a response result output by the digital employee model.

[0081] The execution module 330 is used to execute the pending business instruction based on the response result and obtain the business execution result.

[0082] Among them, the digital employee model is obtained by training the initial large language model based on sample business instructions and sample response results corresponding to the sample business instructions.

[0083] In some embodiments, the execution module 330 is further used to determine whether the business execution result meets the user's business needs; if the business execution result does not meet the user's business needs, regenerate the response result based on the digital employee model; based on the regenerated response result, execute the pending business instructions again and update the business execution result until the business execution result meets the user's business needs.

[0084] In some embodiments, the execution module 330 is further used to obtain user feedback information; the feedback information is the user's satisfaction score for the response result; based on the feedback information, the digital employee model is reinforced learning training to optimize the digital employee model.

[0085] In some embodiments, the execution module 330 is further used to supervise and fine-tune the digital employee model to obtain a fine-tuned digital employee model; construct a reward model; and perform reinforcement learning training on the fine-tuned digital employee model based on the reward model and feedback information to optimize the fine-tuned digital employee model.

[0086] In some embodiments, the execution module 330 is also used to obtain a basic data set; the basic data set includes work order data and product document data of the user business; based on the basic data set, multiple business instruction question and answer pairs are constructed; each business instruction question and answer pair includes a business instruction and a response result corresponding to the business instruction; the multiple business instruction question and answer pairs are input into a pre-trained synonymous sentence conversion model to perform synonymous sentence conversion, and multiple synonymous business instruction question and answer pairs output by the synonymous sentence conversion model are obtained; each synonymous business instruction question and answer pair includes a business instruction and a response result corresponding to the business instruction; based on multiple business instruction question and answer pairs and multiple synonymous business instruction question and answer pairs, a training sample set is constructed; the training sample set includes multiple sample business instructions and a sample response result corresponding to each sample business instruction; based on the training sample set, the initial large language model is trained to obtain an initial digital employee model; the initial digital employee model is fine-tuned by LoRA to obtain a digital employee model.

[0087] In some embodiments, the execution module 330 is further used to determine a target network layer of the initial digital employee model; based on the low-rank approximation method, add trainable parameters to the target network layer; based on the training sample set, fine-tune the target network layer to obtain the digital employee model.

[0088] The present invention also provides an electronic device. Figure 4 Schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 4 As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other via the communication bus 440. The processor 410 may call logic instructions in the memory 430 to execute the digital employee application method.

[0089] Furthermore, the logic instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0090] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the digital employee application method provided by the above methods is implemented.

[0091] The present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the digital employee application methods provided by the above methods.

[0092] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0093] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0094] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A digital employee application method, characterized in that: include: Obtain the user's pending business instructions; Inputting the to-be-executed business instruction into the digital employee model, and obtaining a response result output by the digital employee model; Based on the response result, executing the to-be-executed business instruction to obtain a business execution result; The digital employee model is obtained by training an initial large language model based on sample business instructions and sample response results corresponding to the sample business instructions.

2. The digital employee application method according to claim 1, characterized in that: After executing the to-be-executed service instruction based on the response result and obtaining the service execution result, the method further includes: Determining whether the service execution result meets the service needs of the user; If the business execution result does not meet the business needs of the user, regenerate a response result based on the digital employee model; Based on the regenerated response result, the to-be-executed service instruction is executed again, and the service execution result is updated until the service execution result meets the service demand of the user.

3. The digital employee application method according to claim 1, characterized in that: After executing the to-be-executed service instruction based on the response result and obtaining the service execution result, the method further includes: Obtaining feedback information from the user; the feedback information is a satisfaction score of the user to the response result; Based on the feedback information, reinforcement learning training is performed on the digital employee model to optimize the digital employee model.

4. The digital employee application method according to claim 3, characterized in that: The step of performing reinforcement learning training on the digital employee model based on the feedback information to optimize the digital employee model includes: Performing supervised fine-tuning on the digital employee model to obtain a fine-tuned digital employee model; Build a reward model; Based on the reward model and the feedback information, reinforcement learning training is performed on the fine-tuned digital employee model to optimize the fine-tuned digital employee model.

5. The digital employee application method according to claim 1, characterized in that: Before obtaining the user's to-be-executed business instruction, the method further includes: Obtaining a basic data set; the basic data set includes work order data and product document data of the user's business; Based on the basic data set, construct a plurality of business instruction question-answer pairs; each of the business instruction question-answer pairs includes a business instruction and a response result corresponding to the business instruction; Inputting the plurality of business instruction question-answer pairs into a pre-trained synonymous sentence conversion model to perform synonymous sentence conversion, and obtaining a plurality of synonymous business instruction question-answer pairs output by the synonymous sentence conversion model; each of the synonymous business instruction question-answer pairs includes a business instruction and a response result corresponding to the business instruction; Based on the plurality of business instruction question-answer pairs and the plurality of synonymous business instruction question-answer pairs, a training sample set is constructed; the training sample set includes a plurality of sample business instructions and a sample response result corresponding to each of the sample business instructions; Based on the training sample set, an initial large language model is trained to obtain an initial digital employee model; The initial digital employee model is subjected to LoRA fine-tuning training to obtain the digital employee model.

6. The digital employee application method according to claim 5, characterized in that: The performing LoRA fine-tuning training on the initial digital employee model to obtain the digital employee model includes: determining a target network layer for the initial digital employee model; Based on a low-rank approximation method, adding trainable parameters to the target network layer; Based on the training sample set, fine-tuning training is performed on the target network layer to obtain the digital employee model.

7. A digital employee application device, characterized in that: include: The acquisition module is used to obtain the user's pending business instructions; A generating module, used for inputting the to-be-executed business instruction into the digital employee model, and obtaining a response result output by the digital employee model; An execution module, used to execute the to-be-executed business instruction based on the response result to obtain a business execution result; The digital employee model is obtained by training an initial large language model based on sample business instructions and sample response results corresponding to the sample business instructions.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the digital employee application method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the digital employee application method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the digital employee application method according to any one of claims 1 to 6 is implemented.

Images