272 results about "Reward value" patented technology

The invention provides a multi-modal large model task processing method, device and equipment based on reinforcement learning, and the method comprises the steps: employing a multi-modal large model, generating G groups of responses for multi-modal visual task data, carrying out the scoring of each group of responses, obtaining a reward value, carrying out the standardization of the reward value, obtaining a dominance score, building a strategy updating gradient through the dominance score, and carrying out the calculation of the strategy updating gradient. A final result is obtained after expectation reward maximization, model parameter adjustment and multi-round iteration, and an expectation reward objective function comprises an expectation calculation item and KL divergence constraint, so that a current strategy can be updated by comparing the performance of candidate strategies through group relative strategy optimization, and local optimum can be helped to be jumped out; the strategy updating amplitude is limited by the KL divergence constraint, and the model is prevented from violently changing in the optimization process, so that the training stability is improved; the dynamic strategy iteration allows the model to adjust the balance between exploration and utilization according to the learning progress on the basis of keeping the stability, thereby further ensuring the effectiveness and stability of strategy optimization.

Disclosed are systems and techniques for training personalized language models. The techniques include applying a plurality of first machine learning models to a first input prompt. Each of the plurality of first machine learning models generates a respective reward value of a first plurality of reward values. The techniques include applying a second machine learning model to the first plurality of reward values to obtain first reward value embeddings; applying a third machine learning model to the first reward value embeddings and the first input prompt to obtain a first output response; calculating a first loss based on a comparison between the first output response and the first input prompt; and causing the second machine learning model to be modified based on the first loss.

The invention relates to the technical field of robot training. The robot self-adaptive training method based on reinforcement learning comprises the steps that task sub-target information is generated through a high-level strategy network, the task sub-target information is input into a low-level execution network, an action control instruction is generated according to the task sub-target information, interaction feedback information is collected in the execution process, and the action control instruction is sent to a robot through a robot. Calculating a reward value according to the interaction feedback information, carrying out association processing on the reward value and the scene complexity parameter, executing a dynamic reward shaping operation, generating an adjusted reward signal, generating a strategy model optimized by meta-learning based on the adjusted reward signal, loading the strategy model in a simulation environment, and carrying out dynamic reward shaping. A target strategy model optimized through simulation training is generated, the target strategy model is loaded to the robot, and the robot is controlled to execute task operation in the actual interaction scene. The method has the effect of realizing adaptive task learning of the robot in a multi-interaction scene.

The invention provides a multi-round automatic machine learning agent system based on reinforcement learning optimization. Comprising a task analysis module used for generating an initial prompt for an MLE agent to call; the MLE agent module is used for generating an executable code; the code executor is used for generating an execution result; the evaluator is used for outputting a normalized value of each index and a code correctness identifier; the reward construction module is used for generating a reward value; the reinforcement learning optimizer is used for calculating group average return and candidate advantages and updating strategy parameters of the MLE intelligent agent module based on the candidate advantages; and the multi-round interaction control module is used for feeding back the execution result of the previous round and the reward value to the MLE agent module in the multi-round interaction process, and controlling code generation of the next round until a preset termination condition is met. According to the invention, strategy adaptive evolution, reinforcement learning optimization of fine-grained credit distribution and multi-round closed-loop automatic process improvement can be realized.

The invention relates to the technical field of artificial intelligence, and provides a large language model optimization method and device. According to the method, a candidate answer is obtained by inputting a question in a pre-trained large language model, a first reward value, a second reward value and a third reward value of the candidate answer are calculated, the first reward value is used for representing a basic reward of the candidate answer, the second reward value is used for representing the integrating degree of the candidate answer and the question, and the third reward value is used for representing the integrating degree of the candidate answer and the question. The third reward value is used for representing the alignment degree of the candidate answers and the user preferences, then performing multi-head fusion on the first reward value, the second reward value and the third reward value to obtain a target reward value, and finally training a pre-trained large language model based on the target reward value to obtain an optimized large language model. The importance coefficients of different levels of reward mechanisms of the large language model are adaptively learned in a multi-head layering manner, so that the model training efficiency is improved, and the generation quality of the trained model is improved.

The invention discloses a large language model training method and device, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence and reinforcement learning. The method comprises the steps of obtaining an initial prediction answer statement corresponding to a question statement; semantic expansion retrieval is carried out on the initial prediction answer statement, and a retrieval answer statement corresponding to the initial prediction answer statement is determined; scoring the retrieval answer statement, and determining a target reward value of the initial prediction answer statement based on a scoring result of the retrieval answer statement; and if the target reward value is smaller than a preset reward threshold value, adjusting the pre-trained large language model based on the target reward value, and inputting the question statement into the adjusted model again until the finally obtained target reward value is greater than or equal to the preset reward threshold value, thereby completing training of the pre-trained large language model. The optimization training efficiency of the large language model and the accuracy of model output can be improved, and the stability of large language model output is improved.

The invention discloses an inner draught fan control method and device, equipment, a medium and a computer program product, and relates to the technical field of inner draught fans, and the inner draught fan control method comprises the steps that a historical reward value obtained after an inner draught fan executes a first state adjustment action of the last time step is obtained; the first state adjustment action is an action determined based on an empirical value in a previous preset empirical value table; on the basis of the historical reward value and the environment state of the inner fan in the current time step, an empirical value in a previous preset empirical value table is updated; and based on a target empirical value corresponding to the environment state of the current time step in the updated current preset empirical value table, controlling the inner fan to execute a second state adjustment action of the current time step so as to adjust the current state parameter value of the inner fan to a target state parameter value corresponding to the target empirical value. By introducing reinforcement learning, the inner fan can continuously interact with the environment and accumulate experience, so that the optimal control scheme under different working conditions is learned, and the self-adaptive control of the state of the inner fan is realized.

The invention discloses an automatic leveling method for intelligently controlling an underwater leveling machine. The method comprises the following steps: acquiring real-time attitude data and working surface flatness requirements of a plurality of execution mechanisms of the underwater leveling machine, and constructing state and operation action information characteristics; an execution mechanism is mapped into an agent with a body, information is shared through an underwater acoustic communication network, and each agent selects and executes a leveling action based on self and neighborhood interaction data; after execution, obtaining a reward value, storing the reward value in an experience pool, sampling and updating the state-action function, and performing iterative training to obtain a multi-body agent reinforcement learning optimization model; continuously monitoring the state of a mechanism when the model runs, dynamically updating the features of an affected area when a fault or hydrodynamic abnormality is identified, and recalculating a leveling strategy to obtain a dynamic re-planning scheme; the scheme is combined with model output to generate a real-time leveling instruction sequence, and the real-time leveling instruction sequence is decomposed and then issued to each execution mechanism, so that self-adaptive accurate leveling control of the underwater leveling machine is realized, and the working efficiency and reliability of a complex underwater environment are effectively improved.

Implementations are directed to generating synthetic labeled / preference data by extracting preference pairs from sets of N outputs to a given unlabeled input to a generative model. A plurality of generative outputs are generated by a generative model from a set of input data. A reward model is used to determine a plurality of reward values for the plurality of generative outputs. Based on the reward values, a pair of generative outputs from the plurality of generative outputs is selected for inclusion in a training example. The pair of outputs include a positive training example and a negative training example, where the reward values indicate that the positive training example is preferred over the negative training example. The process can be repeated for a plurality of sets of input data to generate a plurality of training examples for inclusion in a training dataset, which can be used to update reward model(s).

The invention discloses a reinforcement learning training method and system for relieving illusion of a multi-modal large model, and belongs to the field of reinforcement learning training of a multi-modal large language model. Firstly, a planning and visual description generation step is introduced in an early stage to guide a model to perform structured reasoning, then a grouping relative strategy optimization algorithm is used, reward values are calculated for multiple candidate responses generated by the model after cold start, and particularly, a visual perception reward mechanism is set. The reward mechanism evaluates the consistency of the generated text description and the visual information by using an external large language model. Then, based on a vision description attention score advantage distribution method, learning of the model on key vision signals is dynamically enhanced, and the perception ability of the model on the vision signals is improved; and finally, the perception and reasoning performance of the model is further improved by adopting multiple rounds of rejection sampling and supervised fine tuning. The scheme does not depend on a model architecture, the extra overhead is small, the illusion problem caused by early image-text inconsistency is effectively solved, and the accuracy and the reliability are improved.

The invention relates to a self-adaptive plug-in architecture optimization method and device based on Q-Learning, and the method comprises the steps: defining a state space, an action space and a reward function of a target object, and initializing a mapping table of states and action values; obtaining a preset action selection strategy, and determining and executing a target execution action according to the action selection strategy; obtaining running state information after the target object executes the target execution action, and determining an instant reward value according to the running state information and the reward function; and acquiring a preset action value updating strategy, and iteratively determining the updated action value according to the action value updating strategy, the instant reward value, the state space and the action space until the change amplitude of the updated action value is not greater than a preset amplitude threshold. By adopting the method, the requirement of plug-in configuration can be automatically adjusted according to the change of the system state and the service requirement, so that the stability and the reliability of the system are improved.

The invention relates to the technical field of artificial intelligence, and provides a method and device for generating a task execution result, equipment and a medium. According to the method, a reward evaluation agent set is determined based on task description, and a weight is determined for each reward evaluation agent in the set; and obtaining at least one sampling result based on the task description by using a pre-training strategy model, evaluating each sampling result by using each reward evaluation agent, determining a strategy model reward value based on the weight of each reward evaluation agent and the evaluation result, training the strategy model based on the reward value, and obtaining the reward value of the reward evaluation agent. And finally, determining a task execution result by using the trained strategy model, so that a plurality of appropriate reward evaluation agents are automatically and accurately matched for a non-deterministic task, and the plurality of reward evaluation agents are used for jointly training the strategy model to improve the evaluation accuracy of the strategy model. And thus, a task execution result more conforming to the task description is obtained, and the user experience is improved.

In various embodiments, systems and methods for generating interfaces including interface elements representative of next-best actions are disclosed. A request for an interface including a set of features representative of a user associated with the request is received and a user state representation including an implicit user state representation and an explicit user state representation is generated based on the set of features and session data for at least one session associated with the user. An action reward value for each of a plurality of candidate actions is generated based on the user state representation and an interface including at least one interface element representative of a candidate action having a highest action reward value is generated.

The invention provides a reward model training method and device, a strategy model training method and device, electronic equipment and a storage medium, and belongs to the technical field of artificial intelligence, and the method comprises the steps: obtaining a preference data pair which comprises a preferred response and a non-preferred response generated for the same prompt word, and each of the preferred response and the non-preferred response is composed of a plurality of text unit sequences; inputting each text unit sequence into a to-be-trained reward model to obtain a predicted reward value; and calculating the total training loss according to the predicted reward value, and updating the model parameters of the to-be-trained reward model. According to the method, the response text is subjected to serialized splitting, and the preference data composed of the preferred response and the non-preferred response is introduced for comparative learning, so that the target of model training is no longer to evaluate the absolute quality of a single response, but to identify a key text unit which causes one response to be superior to the other response; the fine-grained evaluation of the response content is realized, and the evaluation accuracy of the reward model and the identification capability of complex user preferences are effectively improved.

The invention belongs to the field of natural language processing, and particularly relates to the field of large language models, in particular to an education large model training method under a fine-grained weighted reward mechanism. The method comprises the steps of collecting vocational education field data training strategy models and cleaning data; after completing the training of the base large model, training a sentence-level fine-grained reward model by using supervision data, and designing a context-associated scoring mechanism; after scoring model training and large model supervision fine tuning are completed, large model alignment is carried out, and the strategy is updated according to the total reward value. According to the method, the context factor and the dimension interaction factor are introduced, so that sentence scoring is more reasonable and effective, the safety and reliability of a large language model in the vocational education field are improved, and the method has wide application prospects.

The invention discloses an internet integral dynamic incentive mechanism design method based on reinforcement learning. The method comprises the following steps: S1, constructing a user behavior data set; s2, generating a situation label by adopting a natural language processing technology; s3, calculating a reward value by using a context-adaptive reward algorithm; s4, dynamically adjusting the reward strategy through the reward value and the user behavior feedback in combination with the task participation condition of the user and a feedback loop; s5, predicting a future behavior trend of the user by using the gating cycle unit model, and generating a behavior prediction result of the user; s6, dynamically optimizing the reward strategy in combination with a chaos search strategy and a wolf pack optimization algorithm to obtain an optimized reward strategy; and S7, evaluating effectiveness of rewards and user interest changes, obtaining a final global reward strategy based on an A3C algorithm, and realizing real-time dynamic adjustment of an internet integral incentive mechanism. According to the method, a situation self-adaptive reward algorithm, an optimization technology and the like are utilized, and the dynamic adjustment of an internet integral incentive strategy is realized.

The invention provides a large language model training method and device and electronic equipment. The method comprises the steps that control information is set for a large language model through system cue words of the large language model; inputting training data to the large language model, obtaining output data of the large language model, and extracting N digits from the output data; obtaining a first reward value according to the number M of the numbers carrying the preset number label and the number source in the N numbers; in M numbers carrying preset number labels and number sources, standard values corresponding to the numbers are determined according to the number sources, and second reward values are obtained according to comparison results of the standard values and the numbers; and performing reinforcement learning training on the large language model by adopting a near-end strategy optimization mode or a group relative strategy optimization mode, and forming a training reward value in the near-end strategy optimization mode or the group relative strategy optimization mode according to the first reward value and the second reward value. According to the embodiment of the invention, the accuracy of digits generated by the large language model can be improved.

The invention provides a method and equipment for distributing resource data in a game, and the method comprises the steps: collecting high-dimensional data and game state data in the game, and extracting feature data in the high-dimensional data through an attention mechanism; jointly constructing a behavior prediction model based on the user side and the server side, inputting the feature data into the behavior prediction model, and obtaining behavior data of the player; based on the game state data and the behavior data, utilizing a reward function to calculate a reward value of the player; dynamically adjusting an allocation strategy of resources in the game according to the reward value; and monitoring an effect of resource allocation in the game in real time, and performing iterative optimization on the behavior prediction model according to a monitoring result. According to the method, the reward value is innovatively introduced as a distribution basis, and accurate putting of resources in the game is realized through sorting and priority setting. According to the strategy, the use efficiency of resources is improved, the enthusiasm and participation degree of players are stimulated, and the satisfaction degree of the players is improved.

The invention relates to the technical field of artificial intelligence, in particular to a method, a system and equipment for constructing a primary and middle school education tutoring large model and a medium. The method comprises the following steps: firstly, taking Qwen2.5-3B-Instruct as a basic model, and constructing a training data set according to an obtained teaching data set; then, according to the training data set, constructing a reward function to obtain a reward value, and calling a reinforcement learning method to adjust model parameters to obtain a reinforced basic model; and finally, according to the constructed teacher style data set and the set personalized cue words, the strengthened basic model is finely adjusted in a supervision instruction fine adjustment mode, a middle and primary school education tutoring large model is obtained, through two-stage fine adjustment, the reasoning ability of the model is improved, and the calculation efficiency is improved while the low calculation resource requirement is kept. The processing capability of the old people model for complex problems is improved; and a clear teacher style is endowed to the old people model.

A method and apparatus for preference-training a language model are provided. The method according to some embodiments may include obtaining a dataset including a plurality of pieces of response data, wherein each of the plurality of pieces of response data includes multiple responses generated by the language model for a query and user preference information corresponding to each of the multiple responses, filtering out some of the plurality of pieces of response data included in the dataset using reward values for each of the plurality of pieces of response data, output from a proxy model that receives the dataset as input and training the language model using other pieces of response data that have not been filtered out.

The invention provides a task allocation system training method and device and a readable storage medium, and the method comprises the steps: carrying out the modeling processing of an ISCC task allocation scene, and obtaining an ISCC task allocation system; repeatedly executing the target process until the first reward value of the first reward function and the second reward value of the second reward function respectively reach the maximum value; the target process comprises the following steps: distributing a high-level task to a low-level agent according to a high-level agent algorithm; according to a low-layer agent algorithm, resource information contributed by the low-layer agent to the target task is determined, feedback information is sent to the high-layer agent, a second reward value is obtained, and a second network parameter in the low-layer agent algorithm is updated according to the second reward value; obtaining the first reward value, and updating a first network parameter in the high-level agent algorithm according to the first reward value; the first reward value is determined according to feedback information of the low-layer agent.

The embodiment of the invention provides a reinforcement learning training method and device for a large language model, and the method comprises the steps: inputting a first input text into the large language model, enabling the large language model to process the first input text based on a current strategy, and obtaining G output texts; respectively combining the G output texts with the first input text, and respectively inputting combination results into a trained reward model to obtain G reward values corresponding to the G output texts; the current strategy is updated based on a target function, the target function comprises a global penalty term, the global penalty term represents that a penalty value is applied only when the maximum value in the G reward values does not exceed a preset threshold value, and the penalty value is determined according to respective first output probabilities of the G output texts under the current strategy; therefore, the training of the large language model under the reinforcement learning framework is realized.

In some aspects, the techniques described herein relate to a method including: generating, by a reinforcement learning engine, a plurality of user states within a simulated 3-dimensional environment; generating, by the reinforcement learning engine and for each of the plurality of user states, a plurality of movement actions, wherein each of the plurality of movement actions place a virtual content object in a corresponding position within the simulated 3-dimensional environment; generating, by the reinforcement learning engine and for each corresponding position of the virtual content object, a reward value, wherein the reward value reflects a relative utility of the corresponding position of the virtual content object; and updating, by the reinforcement learning engine, a placement policy based on the reward value for each corresponding position of the virtual content object.

According to the model training method and device, the task execution method and device, the electronic equipment and the storage medium, in the method, the prompt content, the reply content output by the target model for the prompt content and the process data for generating the reply content can be firstly obtained, and then the prompt content, the reply content and the process data are input into the reward system; therefore, the reward value of each reasoning step in the process data is obtained, and finally, the target model is iteratively trained based on the reward value of each reasoning step. The reward system of the method no longer generates the reward value for the token of the sample, but generates the reward value for each reasoning step in the process data, so that the target model can pay attention to the integrity and logicality of the reply content in the training process, and then the performance and stability of the target model in a complex task and the robustness of the model are improved.

The invention relates to the technical field of intelligent questioning and answering, and provides an intelligent questioning and answering method and system. The method comprises the steps of performing emotion recognition on current question content to obtain an emotion type and emotion intensity; adjusting the current prompt sentence based on the emotion type and the emotion intensity to obtain a target prompt sentence; generating current answer content based on the target prompt sentence and the current question content; and generating a reward value of the current answer content based on the reward function, and if the reward value is not converged, returning to the step of adjusting the mood prompt and the word prompt in the current prompt sentence based on the emotion type and the emotion intensity to obtain the target prompt sentence until the reward value is converged, and determining the current answer content at the moment as the target answer content. The emotion tendency of the user question can be effectively identified, and the answer content is generated and adjusted based on the emotion tendency of the user, so that the target answer content meets the emotion demand of the user to the greatest extent while accurately answering the user question, and the user experience is improved.

A reinforcement learning-based system for adaptively controlling computing performance is provided. The system includes an environment module and an agent module. The environment module is configured to collect environment information from the application environment. Based on the collected environment information, the environment module calculates a reward value using a reward function and then outputs the state data and the reward value to the agent module. The agent module receives the output from the environment module, including the reward value and the state data. Based on the received reward value and state data, the agent module determines a performance adjustment action to take, which is then fed back to the application environment. Upon receiving the performance adjustment action, the application environment executes the performance adjustment operation in response, causing the environment module to collect the updated environment information as a result of the performance adjustment operation.

The invention provides a reinforcement learning training method and system based on a multi-target structured reward function, and the method comprises the steps: an environment simulation layer comprises a physical engine and a blood vessel environment simulator, and is used for simulating the physical movement and interaction of a catheter in a blood vessel, and providing an environment state for a reward calculation layer; the reward calculation layer performs calculation through a plurality of independent calculation modules and finally generates a comprehensive reward value through a composite reward aggregator; the reinforcement learning layer learns and optimizes a navigation strategy according to the comprehensive reward value; and the monitoring evaluation layer monitors each performance index in the training process and adaptively adjusts the weight of each reward component in the reward calculation layer according to the index change. According to the invention, a major breakthrough of reinforcement learning in medical robot training is realized. Compared with a traditional single reward function method, the intelligent agent which is really thought by experts can be trained, and the method not only pursues to quickly reach a target, but also knows how to safely and stably complete operation.

There is described methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a reinforcement learning system. The reinforcement learning system comprises an agent configured to perform actions based upon a policy and an intrinsic reward system configured to generate intrinsic reward values for the agent based upon the actions taken by the agent. The method comprises training the reinforcement learning system based upon a plurality of tasks. The training comprises updating the agent's policy based upon the intrinsic reward values generated by the intrinsic reward system and updating the intrinsic reward system based upon an extrinsic reward value obtained based upon the task being performed by the agent. The training further comprises re-initializing the agent's policy when an expiration criterion associated with the agent is met.

The invention discloses an opportunity fair federal learning excitation method and system based on the game theory, and the method comprises the steps: a model owner determines a reward value of a current round, determines the contribution rate of participating in the training of the current round according to each data node of the reward value of the current round, and carries out the local model training corresponding to each data node; the model owner aggregates the local models of the data nodes according to the reputation values of the data nodes to obtain a global model; constructing a first utility function according to the overhead of each data point in the current round, and constructing a second utility function according to the earnings of the model owner; and calculating the utility of a model owner and each data node according to the first utility function and the second utility function, and adjusting the contribution rate of the next round of training and the reward value of the next round according to the utility result. The optimal balance between the power data node retraining contribution rate and the training cost is realized, and the training effect and the duration of federal learning are ensured.

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, performing reward training of a diffusion neural network.