Decision model training method and device and information processing method and device

Abstract

The embodiment of the invention provides a decision model training method and device and an information processing method and device.The decision model training method comprises the steps that according to a preset decision data construction strategy, sample data associated with a target language model is processed into sample sequence data; extracting sample state data and sample action data from the sample sequence data, and inputting the sample state data and the sample action data into the target language model for processing to obtain prediction reward data; using sample reward data in the sample sequence data and the prediction reward data to update the target language model into an initial decision model; and processing service data corresponding to a service task into service sequence data according to the decision data construction strategy, and optimizing the initial decision model into a service decision model corresponding to the service task by using the service sequence data.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of machine learning technology, and in particular to a decision model training method and device, and an information processing method and device. Background Art

[0002] With the development of the Internet and computer technology, decision models are being applied in more and more fields. In specific decision-making situations, decision models can be used to analyze and solve problems by establishing mathematical or logical models. They are mainly used to describe and solve decision-making problems, helping decision makers make optimal or satisfactory decisions in complex environments. For example, the advertising delivery system of e-commerce platforms, search engine optimization tools, game AI decision systems, robot path planning, etc., can all use decision models to complete the prediction of decision information for different tasks in the execution phase. In the prior art, traditional decision models, such as expert systems, planning algorithms, and reinforcement learning, perform well in specific application tasks, but when faced with new or unforeseen tasks, they have poor generalization capabilities and cannot provide stable decision-making services. Therefore, an effective solution is urgently needed to solve the above problems. Summary of the invention

[0003] In view of this, an embodiment of this specification provides a decision model training method. One or more embodiments of this specification also relate to a decision model training method, an information processing method, an information processing device, a computing device, a computer-readable storage medium, and a computer program product to solve the technical defects existing in the prior art.

[0004] According to a first aspect of an embodiment of this specification, a decision model training method is provided, including:

[0005] According to a preset decision data construction strategy, the sample data associated with the target language model is processed into sample sequence data;

[0006] Extracting sample state data and sample action data from the sample sequence data and inputting them into the target language model for processing to obtain predicted reward data;

[0007] Using the sample reward data and the predicted reward data in the sample sequence data, updating the target language model to an initial decision model;

[0008] The service data corresponding to the service task is processed into service sequence data according to the decision data construction strategy, and the initial decision model is optimized into a service decision model corresponding to the service task using the service sequence data.

[0009] According to the second aspect of the embodiment of this specification, there is provided an information processing method, including:

[0010] Receive service information submitted by the client for the target service;

[0011] Inputting the service information into a service decision model associated with the target service for processing to obtain service decision information, wherein the service decision model is obtained by training using the above method;

[0012] The service decision information is fed back to the client.

[0013] According to a third aspect of an embodiment of this specification, another decision model training method is provided, which is applied to a cloud-side device, including:

[0014] According to a preset decision data construction strategy, the sample data associated with the target language model is processed into sample sequence data;

[0015] Extracting sample state data and sample action data from the sample sequence data and inputting them into the target language model for processing to obtain predicted reward data;

[0016] Using the sample reward data and the predicted reward data in the sample sequence data, updating the target language model to an initial decision model;

[0017] In the case of receiving an optimization request submitted by the end-side device for the initial decision model, determining service data according to the optimization request;

[0018] The service data is processed into service sequence data according to the decision data construction strategy, and the initial decision model is optimized into a service decision model using the service sequence data.

[0019] According to a fourth aspect of the embodiments of this specification, a decision model training device is provided, including:

[0020] A strategy processing module is configured to construct a strategy according to preset decision data and process sample data associated with the target language model into sample sequence data;

[0021] A model processing module is configured to extract sample state data and sample action data from the sample sequence data and input them into the target language model for processing to obtain predicted reward data;

[0022] An updating model module is configured to update the target language model to an initial decision model using the sample reward data and the predicted reward data in the sample sequence data;

[0023] The optimization model module is configured to process the service data corresponding to the service task into service sequence data according to the decision data construction strategy, and optimize the initial decision model into a service decision model corresponding to the service task using the service sequence data.

[0024] According to a fifth aspect of the embodiments of this specification, there is provided an information processing device, including:

[0025] An information receiving module is configured to receive service information submitted by a client for a target service;

[0026] An information processing module is configured to input the service information into a service decision model associated with the target service for processing to obtain service decision information, wherein the service decision model is obtained by training using the above method;

[0027] The information sending module is configured to feed back the service decision information to the client.

[0028] According to a sixth aspect of an embodiment of this specification, another decision model training device is provided, which is applied to a cloud-side device, including:

[0029] A processing module is configured to process the sample data associated with the target language model into sample sequence data according to a preset decision data construction strategy;

[0030] An input module is configured to extract sample state data and sample action data from the sample sequence data and input them into the target language model for processing to obtain predicted reward data;

[0031] An updating module, configured to update the target language model to an initial decision model using the sample reward data and the predicted reward data in the sample sequence data;

[0032] A determination module, configured to determine service data according to the optimization request submitted by the terminal device for the initial decision model when receiving the optimization request;

[0033] The optimization module is configured to process the service data into service sequence data according to the decision data construction strategy, and optimize the initial decision model into a service decision model using the service sequence data.

[0034] According to a seventh aspect of an embodiment of this specification, there is provided a computing device, including:

[0035] Memory and processor;

[0036] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the above-mentioned decision model training method or information processing method are implemented.

[0037] According to an eighth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the above-mentioned decision model training method or information processing method are implemented.

[0038] According to the ninth aspect of the embodiments of this specification, a computer program product is provided, including a computer program or instructions, which, when executed by a processor, implement the steps of the above-mentioned decision model training method or information processing method.

[0039] The decision model training method provided in this embodiment is to improve the effectiveness and adaptability of the decision model in the decision task and facilitate downstream service applications. A general decision model can be trained first. In this process, the sample data of the associated target language model can be processed into sample sequence data according to the preset decision data construction strategy; at this time, the sample state data and sample action data can be extracted from the sample sequence data and input into the target language model for processing, so that the predicted reward data can be obtained according to the processing results; on this basis, the sample reward data and predicted reward data in the sample sequence data can be used to update the target language model to the initial decision model; the use of the adjusted language model as a decision model can make the model have stronger generalization ability. Thereafter, the service data corresponding to the service task is processed into service sequence data according to the decision data construction strategy, and the initial decision model is optimized to the service decision model corresponding to the service task using the service sequence data. It can be realized that the general decision model is trained as a service decision model for the associated service task, so that the service decision model can have stronger decision-making ability under the service task dimension, and is more adapted to the service environment corresponding to the service task, and is more convenient for downstream service use. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a schematic diagram of a decision model training method provided by an embodiment of this specification;

[0041] Figure 2 is a flow chart of a decision model training method provided by an embodiment of this specification;

[0042] Figure 3 is a flow chart of an information processing method provided by an embodiment of this specification;

[0043] Figure 4is a flow chart of another decision model training method provided by an embodiment of this specification;

[0044] Figure 5 is a processing flow chart of a decision model training method provided by an embodiment of this specification;

[0045] Figure 6 It is a structural schematic diagram of a decision model training device provided by an embodiment of this specification;

[0046] Figure 7 is a structural diagram of an information processing device provided by an embodiment of this specification;

[0047] Figure 8 It is a structural diagram of another decision model training device provided by an embodiment of this specification;

[0048] Fig. 9 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0049] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0050] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0051] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0052] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0053] In one or more embodiments of this specification, a large model refers to a deep learning model with large-scale model parameters, which usually contains hundreds of millions, tens of billions, hundreds of billions, trillions, or even more than 10 trillion model parameters. A large model can also be called a foundation model / foundation model. The large model is pre-trained with large-scale unlabeled corpus to produce a pre-trained model with more than 100 million parameters. This model can adapt to a wide range of downstream tasks, and the model has good generalization ability, such as a large-scale language model (LLM), a multi-modal pre-training model, etc.

[0054] When the big model is used in practice, only a small number of samples are needed to fine-tune the pre-trained model and it can be applied to different tasks. The big model can be widely used in natural language processing (NLP), computer vision and other fields. Specifically, it can be applied to computer vision tasks such as visual question answering (VQA), image caption (IC), image generation, as well as natural language processing tasks such as text-based sentiment classification, text summary generation, and machine translation. The main application scenarios of the big model include digital assistants, intelligent robots, search, online education, office software, e-commerce, intelligent design, etc.

[0055] First, the terms involved in one or more embodiments of this specification are explained.

[0056] LLMs: Large Language Models, refers to artificial intelligence models that have a large number of parameters and data and can perform multiple language tasks.

[0057] CT: Continued Training, which refers to continuing training on the basis of a pre-trained large language model to enhance its capabilities in a specific field or language.

[0058] SFT: Supervised Fine-Tuning, supervised fine-tuning, refers to further adjusting the model through supervised learning based on the pre-trained model to adapt to specific downstream tasks.

[0059] NLU: Natural Language Understanding, refers to the ability of machines to understand human language.

[0060] NLG: Natural Language Generation, refers to the ability of machines to generate human language.

[0061] In this specification, a decision model training method is provided. One or more embodiments of this specification simultaneously relate to a decision model training method, an information processing method, an information processing apparatus, a computing device, a computer-readable storage medium, and a computer program product, which are described in detail one by one in the following embodiments.

[0062] See also Figure 1 As shown in the schematic diagram, this embodiment provides a decision model training method, in order to improve the effectiveness and adaptability of the decision model in the decision task, and facilitate downstream service applications. A general decision model can be trained first. In this process, the sample data of the associated target language model can be processed into sample sequence data according to the preset decision data construction strategy; at this time, the sample state data and sample action data can be extracted from the sample sequence data and input into the target language model for processing, so that the predicted reward data can be obtained according to the processing results; on this basis, the sample reward data and predicted reward data in the sample sequence data can be used to update the target language model to the initial decision model; the use of the adjusted language model as a decision model can make the model have stronger generalization ability. Thereafter, the service data corresponding to the service task is processed into service sequence data according to the decision data construction strategy, and the initial decision model is optimized to the service decision model corresponding to the service task using the service sequence data. It can be realized that the general decision model is trained as a service decision model for the associated service task, so that the service decision model can have stronger decision-making ability under the service task dimension, and is more adapted to the service environment corresponding to the service task, and is more convenient for downstream service use.

[0063] See also Figure 2 , Figure 2 A flowchart of a decision model training method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0064] Step S202: Process the sample data associated with the target language model into sample sequence data according to a preset decision data construction strategy.

[0065] The decision model training method provided in this embodiment can be applied to decision model training in any scenario, such as e-commerce platform advertising delivery system (the model is used to predict the click-through rate and cost per click of specific advertisements), search engine optimization work (the model is used to predict the number of product impressions and click-through rate), game AI decision system (the model is used to predict the game decision control type) or robot path planning (the model is used to predict the robot's driving behavior), etc. When it comes to the decision model required for the decision-making task, the decision model can be trained using the decision model training method provided in this embodiment, so that the decision model can give accurate decision information when performing the decision-making task in the corresponding scenario.

[0066] Specifically, the decision data construction strategy refers to a strategy for processing sample data to obtain sample sequence data. After processing the data based on the strategy, the processed samples can be used to update the target language model to an initial decision model with general decision-making capabilities, so that it can be optimized for specific tasks later. In specific implementation, the decision data construction strategy can be set according to the service scenario requirements. When setting the decision data construction strategy under the target service scenario, the decision data construction strategy can be set as a rule for constructing sample data associated with the target service scenario into a SAR (sample state data, sample action data, and sample reward data) structure; when reconstructing the sample data corresponding to the target service scenario, it can be processed into a SAR structure, so that when the model is trained later, the model can learn decision-making capabilities.

[0067] It can be understood that when the decision data construction strategy processes the sample data, it can implement the reconstruction operation of the sample data so that the sample data can be changed into sample sequence data, which is composed of sample state data, sample action data and sample sequence data. Correspondingly, the sample data specifically refers to the samples recorded in the sample set corresponding to the target language model. Correspondingly, the target language model specifically refers to a large language model of any architecture, which can be selected according to actual needs, and this embodiment does not make any limitation here. Correspondingly, the sample sequence data specifically refers to the data obtained after the sample data is reconstructed according to the decision data construction strategy, and the data is composed of sample state data, sample action data and sample sequence data.

[0068] Among them, the sample state data can be represented by S (State), which contains decision background information, task-related details, and information needed to make decisions; correspondingly, the sample action data can be represented by A (Action), which contains specific measures under a given state, such as virtual actions in the game (such as controlling the flight decision of the game character) or decision actions in reality (such as taking the bus or buying books). It should be noted that the sample action data needs to be converted into text format for use in model training. Correspondingly, the sample reward data can be represented by R (Reward), which contains the feedback received after taking the corresponding action in a specific state. It can be a quantitative measure, such as user ratings, or qualitative feedback (such as "I like it" or "He is satisfied with it").

[0069] In practical applications, considering that the traditional large language model is usually modeled in the form of next token prediction (NextTokenPrediction), this training form does not consider the different components of the text or article itself (such as title, beginning, and end), and the extraction of different components implied in the context or sequence is crucial for training the decision model. Therefore, in order to be able to update the large language model to a general decision model so that it can be subsequently tuned to the service decision model corresponding to the specified task, the decision data construction strategy will be used to process the sample data into sample sequence data, so that the target language model can have decision-making capabilities through the sample sequence data.

[0070] Based on this, in order to improve the effectiveness and adaptability of the decision model in the decision task and facilitate downstream service applications. A general decision model can be trained first. In this process, the sample data of the associated target language model can be processed into sample sequence data according to the preset decision data construction strategy; at this time, the sample state data and sample action data can be extracted from the sample sequence data and input into the target language model for processing, so that the predicted reward data can be obtained according to the processing results; on this basis, the sample reward data and predicted reward data in the sample sequence data can be used to update the target language model to the initial decision model; the use of the adjusted language model as the decision model can make the model have stronger generalization ability. Thereafter, the service data corresponding to the service task is processed into service sequence data according to the decision data construction strategy, and the initial decision model is optimized to the service decision model corresponding to the service task using the service sequence data. It can be realized that the general decision model is trained as the service decision model of the associated service task.

[0071] Furthermore, when processing sample data according to the decision data construction strategy, it is considered that the sample data corresponding to the target language model may contain content of different richness. This may involve single-step decisions or sequence decisions when training the target language model. Therefore, when processing it into sample sequence data, different processing is required according to different types.

[0072] In this embodiment, the specific implementation method is as follows:

[0073] Acquire sample data associated with the target language model; and test the sample data according to a preset decision data construction strategy;

[0074] (1) When it is determined according to the detection results that the sample data is a single-step decision data type, the sample data is processed according to the decision data construction strategy; sample state data, sample action data and sample reward data are determined according to the processing results, and sample sequence data is constructed based on the sample state data, the sample action data and the sample reward data.

[0075] Specifically, the single-step decision data type refers to the type of sample data involving a single-step decision problem. At this time, it is not complicated to determine its data structure. Therefore, it is directly processed according to the decision data construction strategy to construct the sample data into sample sequence data consisting of three parts of data.

[0076] Based on this, when using the decision data construction strategy to process the sample data of the target language model, the sample data associated with the target language model can be obtained first; thereafter, the sample data can be detected according to the preset decision data construction strategy; if the sample data is determined to be a single-step decision data type according to the detection result, it means that the sample data involves a single-step decision problem, which does not contain a more complex decision problem structure, so the sample data can be directly processed according to the decision data construction strategy; the sample state data, sample action data and sample reward data are determined according to the processing results, and at this time, the sample sequence data corresponding to the sample data can be constructed based on the sample state data, sample action data and sample reward data for subsequent use.

[0077] This embodiment describes the application of the decision model training method in the advertising delivery system as an example. The description of other scenarios can refer to the same or corresponding description in this embodiment, and this embodiment will not be repeated here. Specifically, when training a decision model that needs to be applied to the advertising delivery system, the large language model LLM1 as the model base can be determined first, and then sample data can be selected. After that, the sample data is tested according to the decision data construction strategy, and it is determined that the sample data is a single-step decision problem. At this time, the sample data can be processed into a (S, A, R) structure.

[0078] Among them, S represents sample state data, which contains decision background, decision task details, and information required for decision making {Background: I am a professional focusing on advertising placement on e-commerce platforms, mainly responsible for attracting traffic through advertising products. At first, I will pick products with potential...; Title: A brand Android projector; Category: Home audio and video; Attributes: Optical resolution: 1280*720dpi; Bluetooth compatible: yes; Remote control included: yes...}. A represents sample action data, which contains specific measures in a given state {Currently showing a white projector with 4K resolution, which is designed for high-quality video and gaming experience...}. R represents sample reward data, which contains feedback received after the action {After one month of advertising promotion, the following feedback was received: click-through rate (CTR), cost per click (CPC), impressions (Impressions)...}. After that, the large language model LLM1 can be trained in combination with the above information, so that the model can learn the intrinsic patterns and statistical association knowledge in the decision-making process.

[0079] (2) When it is determined according to the detection result that the sample data is a type of sequence decision data, the decision step information of the sample data is detected according to the decision data construction strategy; the sample data is divided into at least two sample sub-data based on the decision step information; the at least two sample sub-data are processed separately according to the decision data construction strategy, and sample sequence data is constructed according to the at least two processed sample sub-data.

[0080] Specifically, the sequence data type specifically refers to the type of sample data involving a sequence decision problem. At this time, it is more complicated to determine its data structure, so it is necessary to first determine the decision step information so that it can be processed according to the decision step information later. Correspondingly, the decision step information specifically refers to the value recorded when the sample data is divided. This information determines the number of sample sub-data obtained by dividing the sample data. Each sample sub-data can be understood as a separate part. After using the decision data to build a strategy to process it, the sample sub-state data, sample sub-action data and sample sub-reward data corresponding to the sample sub-data can be obtained. By merging the sample sub-sequence data corresponding to each sample sub-data, the sample sequence data can be obtained.

[0081] Based on this, when the sample data is determined to be a sequence decision data type according to the detection result, it means that the sample data has a relatively complex decision data structure, so the decision step information of the sample data can be detected according to the decision data construction strategy first; at this time, the sample data can be divided into at least two sample sub-data based on the decision step information; then, the at least two sample sub-data can be processed separately according to the decision data construction strategy, wherein the processing process of any one sample sub-data is similar to the processing process of the above-mentioned sample data, and this embodiment will not be described in detail here. On this basis, the sample sequence data can be constructed according to the at least two processed sample sub-data for subsequent use.

[0082] Continuing with the above example, when the sample data is determined to be a sequential decision problem, the sample data can be processed as (S1, A1, R1; S2, A2, R2…S n ,A n ,R n ) structure, where n represents the number of decision steps. On this basis, the decision data is constructed for the sample data according to the result, and the sample sequence data composed of the sample subsequence data corresponding to each step can be obtained, so as to train the large language model LLM1 in the future, so that the model can learn the internal patterns and statistical association knowledge in the decision process.

[0083] In summary, when constructing decision data for different types of sample data, the sample data can be processed into sample state data, sample action data, and sample reward data, so that single-step decision problems and sequence decision problems can be modeled. This allows the model to not be specific to any single task during the training phase, but to learn a wider range of decision-making knowledge, thereby realizing the ability to convert the language model into a general decision model.

[0084] Step S204, extracting sample state data and sample action data from the sample sequence data and inputting them into the target language model for processing to obtain predicted reward data.

[0085] Specifically, after the sample data is processed into sample sequence data as mentioned above, since the sample sequence data contains sample state data, sample action data and sample reward data, in order to be able to convert the target language model into a decision model with decision-making ability, the sample state data and sample action data can be extracted from the sample sequence data and input into the target language model for processing, so that the language model can output predicted reward data. Thereafter, the target language model is adjusted in combination with the sample reward data in the sample sequence data, so that the target language model can learn decision-making knowledge.

[0086] In practical applications, the target language model can be trained using the following training formula (1) when learning decision-making knowledge:

[0087]

[0088] Where N represents the number of samples, and each sample is a text sentence or paragraph; j represents the jth sample, represents the i-th word in the j-th sample, Represents all the word units that appear before the j-th word unit in the j-th sample.

[0089] Step S206: using the sample reward data in the sample sequence data and the predicted reward data, the target language model is updated to an initial decision model.

[0090] Specifically, after using the target language model to process the sample state data and sample sequence data as mentioned above, the predicted reward data output by the target language model will be obtained. On this basis, combined with the sample reward data and the predicted reward data in the sample sequence data, the target language model can be converted. After that, through continuous iterative processing, the target language model can be updated to the initial decision model. After obtaining the initial decision model with general decision-making capabilities, it can be fine-tuned under the specified task to obtain the decision model for the corresponding service scenario.

[0091] In practical applications, after the sample state data and sample action data are input into the target language model for processing, the target language model is actually used to predict the reward probability distribution corresponding to the given state and action of the sample. After that, for each decision task, the action with the largest reward can be selected to execute, so that during the training phase of the target language model, the model can predict the reward probability, so as to subsequently combine the predicted reward probability with the sample reward data contained in the sample sequence data to complete the parameter adjustment of the target language model, and iterate the processing to form the initial decision model for subsequent fine-tuning.

[0092] In practical applications, when updating the target language model, in order to enable the updated target language model to have decision-making capabilities, the conversion from the language model to the decision model can be completed by iterative parameter adjustment. In this embodiment, the specific implementation method is as follows:

[0093] Extracting sample reward data from the sample sequence data, and calculating a target loss value based on the sample reward data and the predicted reward data; adjusting the parameters of the target language model according to the target loss value, and detecting whether the adjusted target language model meets the training stop condition; if not, selecting candidate sample data associated with the target language model as the sample data, and executing a step of processing the sample data associated with the target language model into sample sequence data according to a preset decision data construction strategy; if so, using the adjusted target language model as the initial decision model.

[0094] Based on this, after obtaining the predicted reward data output by the target language model after processing the sample state data and sample action data, the sample reward data can be extracted from the sample sequence data. At this time, the target loss value is calculated based on the sample reward data and the predicted reward data; and the target language model can be adjusted according to the target loss value.

[0095] After the parameter adjustment is completed, it can be detected whether the adjusted target language model meets the training stop condition. If not, it means that the target language model at the current stage does not have a good decision-making ability, so the candidate sample data associated with the target language model can be selected as the sample data, and step S202 is executed. Through continuous training, until the training stop condition is met, the adjusted target language model can be used as the initial decision model.

[0096] During specific implementation, the training stop condition can be set according to actual needs, such as the number of iterations, verification conditions, etc., and this embodiment does not make any limitation here.

[0097] Continuing with the above example, after obtaining the SAR data structure corresponding to the sample data, the sample state data and sample action data in the SAR data structure can be input into the large language model LLM1 for processing, so that the model can output the reward probability {predict the feedback after one month of advertising promotion: predict click-through rate (CTR), predict cost per click (CPC), predict impressions (Impressions)...}. After that, the large language model LLM1 can be adjusted in combination with the sample reward data corresponding to the sample data. If it is determined that the current number of iterations is less than M after parameter adjustment, new sample data can be selected in the data set and the above training can be repeated. When the number of iterations reaches M, the large language model obtained by the final parameter adjustment can be used as a general decision model. This enables the model to have decision-making capabilities so that it can be trained and used for specific tasks in the future.

[0098] In summary, by converting the target language model into a decision model, the generalization ability of the language model can be applied to decision-making tasks, thereby effectively improving the effectiveness and adaptability of the decision model.

[0099] Step S208: Processing the service data corresponding to the service task into service sequence data according to the decision data construction strategy, and optimizing the initial decision model into a service decision model corresponding to the service task using the service sequence data.

[0100] Specifically, after converting the target language model into the initial decision model as mentioned above, in order to enable the initial decision model to have stronger decision-making ability in a specific service scenario, the service data corresponding to the service task can be processed into service sequence data according to the decision data construction strategy. At this time, the service sequence data is used to optimize the initial decision model, and the service decision model corresponding to the service task can be obtained according to the optimization result, so that the service decision model can complete the decision prediction of the specified task in the scenario corresponding to the service task.

[0101] Among them, service tasks specifically refer to tasks corresponding to different service scenarios, such as advertising delivery tasks in advertising delivery scenarios, game character control tasks in game scenarios, etc. Correspondingly, service data specifically refers to data associated with service scenarios, which can enable the model to learn knowledge corresponding to the scenario, so that the decision model can be used in the scenario corresponding to the service task. Correspondingly, service sequence data specifically refers to sequence data obtained after processing the service data according to the decision data construction strategy; wherein, the processing of service sequence data can refer to the processing process of the above-mentioned sample data, and this embodiment does not make any restrictions here. Correspondingly, the service decision model specifically refers to a decision model that matches the service task scenario, which can be used to predict decision information in the scenario corresponding to the service task. For example, prediction of advertising delivery effects, prediction of game character control, etc.

[0102] Furthermore, since the initial decision model is a decision model obtained by training the target language model, in order to enable the model to have the service decision-making ability of the specified task in the corresponding scenario of the service task, it is necessary to process the service data into service sequence data in the model optimization stage, so as to complete the optimization of the model. In this embodiment, the specific implementation method is as follows:

[0103] When a service task triggered by the initial decision model is detected, service data associated with the service task is obtained; the service data is processed according to the decision data construction strategy to obtain service status data, service action data and service reward data; and service sequence data is constructed based on the service status data, the service action data and the service reward data.

[0104] Specifically, service status data refers to information including service decision background information, service task-related details, and service information required for decision making. Service action data refers to specific measures taken under a given service state. In other words, service reward data refers to feedback received after taking corresponding actions under a specific state.

[0105] Based on this, when a service task is detected that is triggered for the initial decision model, the service data associated with the service task can be obtained at this time; since the training phase of the initial decision model reconstructs the data into states, actions, and rewards, the service data will also be processed in the same way. At this time, the service data can be processed according to the decision data construction strategy to obtain service state data, service action data, and service reward data; on this basis, service sequence data can be constructed based on the service state data, service action data, and service reward data for subsequent training of the initial decision model.

[0106] In summary, when training a decision model that matches a service scenario for a service task, processing the service data according to the same strategy can enable the model to learn the service knowledge corresponding to the service scenario based on the original decision-making ability, thereby training a service decision model that meets the service scenario based on a small number of samples.

[0107] Furthermore, after obtaining the service sequence data, the initial decision model can be optimized in the following manner. In this embodiment, the specific implementation is as follows:

[0108] The service status data and service action data in the service sequence data are input into the initial decision model for processing to obtain predicted service reward data; the initial decision model is optimized based on the service reward data in the service sequence data and the predicted service reward data; and a service decision model that meets the service training stop condition and matches the service task is generated according to the optimization result, wherein the service decision model is used to predict service decision information for the service decision tasks associated with the service task.

[0109] Based on this, in order to enable the initial decision model to learn the service decision-making capabilities of the service scenario corresponding to the service task, the service status data and service action data in the service sequence data can be input into the initial decision model for processing, and the predicted service reward data can be obtained according to the processing results; at this time, the initial decision model can be optimized based on the service reward data and the predicted service reward data in the service sequence data; and the optimized model can be tested. If it is detected that the optimized model does not meet the service training stop conditions in the current service scenario, a new sample can be selected to continue the above processing. Until it is determined according to the optimization results that the service training stop conditions are met, the optimized model at the current stage can be used as the service decision model for matching service tasks, so that the service decision model can be used to predict service decision information for service decision tasks associated with service tasks.

[0110] Continuing with the above example, after obtaining a decision model with general decision-making capabilities, determine the decision model that needs to be trained in the e-commerce advertising task. At this time, you can select the service data corresponding to the e-commerce advertising task to continue training the decision model. Before training, the service data also needs to be converted into a SAR data structure so that the decision model can be fully trained in the field of e-commerce advertising. The trained decision model can predict the click-through rate and cost per click of a specific advertisement.

[0111] In summary, by using corresponding service data to complete training for a specific field, the model can have stronger predictive ability in that field and better adaptability.

[0112] In addition, in order to ensure that the decision model has a relatively general decision-making ability in the initial decision model training stage, the target language model can be trained using data corresponding to multiple data sources. In this embodiment, the specific implementation method is as follows:

[0113] Determine multiple data sources associated with the target language model; extract the sample data from the multiple data sources, and execute the step of processing the sample data associated with the target language model into sample sequence data according to a preset decision data construction strategy.

[0114] Specifically, multiple data sources refer to sample data storage spaces corresponding to different service scenarios. By extracting data from multiple data sources as sample data for subsequent model training, the model's decision-making ability can be stronger and its generalization ability can be better.

[0115] On this basis, in order to ensure that the service decision model has a strong service decision capability, the service decision model can also be verified by using a verification sample. Specifically, the verification sample associated with the service task can be obtained, and the service decision model can be verified by using the verification sample.

[0116] During the verification process, the verification sample can be input into the service decision model for processing. After the prediction result is obtained, the service decision capability of the service decision model can be determined by comparing the prediction result with the label corresponding to the verification sample.

[0117] This embodiment provides a decision model training method, in order to improve the effectiveness and adaptability of the decision model in the decision task, and facilitate downstream service applications. A general decision model can be trained first. In this process, the sample data of the associated target language model can be processed into sample sequence data according to the preset decision data construction strategy; at this time, the sample state data and sample action data can be extracted from the sample sequence data and input into the target language model for processing, so that the predicted reward data can be obtained according to the processing results; on this basis, the sample reward data and predicted reward data in the sample sequence data can be used to update the target language model to the initial decision model; the use of the adjusted language model as the decision model can make the model have stronger generalization ability. Thereafter, the service data corresponding to the service task is processed into service sequence data according to the decision data construction strategy, and the initial decision model is optimized to the service decision model corresponding to the service task using the service sequence data. It can be realized that the general decision model is trained as a service decision model for the associated service task, so that the service decision model can have stronger decision-making ability under the service task dimension, and is more adapted to the service environment corresponding to the service task, and is more convenient for downstream service use.

[0118] Corresponding to the above method embodiment, this specification also provides an information processing method embodiment, Figure 3 FIG. 1 is a flowchart of an information processing method provided by an embodiment of the present specification. Figure 3 As shown, the method includes:

[0119] Step S302, receiving service information submitted by the client for the target service;

[0120] Step S304, inputting the service information into a service decision model associated with the target service for processing to obtain service decision information, wherein the service decision model is obtained by training using the above method;

[0121] Step S306: Feedback the service decision information to the client.

[0122] Specifically, after the service decision model of the corresponding service scenario is trained by the above-mentioned decision model training method, the service decision model can be deployed to the corresponding service scenario. Thereafter, if the service information submitted by the client is received, the service information can be directly input into the service decision model of the corresponding service scenario for processing, so as to obtain the service decision information and then feed it back to the client, so that the client can perform downstream service processing.

[0123] For example, after the service decision model outputs the predicted click-through rate and cost per click of a specified advertisement for one month, the service provider can adjust the delivery strategy of the specified advertisement based on the information, thereby reducing costs while increasing the exposure rate of the advertisement, thereby increasing advertising revenue.

[0124] Corresponding to the above method embodiment, this specification also provides another decision model training method embodiment, Figure 4 FIG. 2 is a flowchart of another decision model training method provided by an embodiment of the present specification. Figure 4 As shown, the method is applied to a cloud-side device, including:

[0125] Step S402, processing the sample data associated with the target language model into sample sequence data according to a preset decision data construction strategy;

[0126] Step S404, extracting sample state data and sample action data from the sample sequence data and inputting them into the target language model for processing to obtain predicted reward data;

[0127] Step S406, using the sample reward data in the sample sequence data and the predicted reward data, updating the target language model to an initial decision model;

[0128] Step S408, when receiving an optimization request submitted by the end-side device for the initial decision model, determining service data according to the optimization request;

[0129] Step S410: Processing the service data into service sequence data according to the decision data construction strategy, and optimizing the initial decision model into a service decision model using the service sequence data.

[0130] The cloud-side device specifically refers to the cloud server that provides model training services. The terminal-side device specifically refers to the terminal device that has model training requirements. After the cloud-side device trains an initial decision model with general decision-making capabilities, the cloud-side device can determine the decision model that needs to be trained for the specified service by submitting an optimization request. At this time, it can be trained again based on the initial decision model to obtain a service decision model for the corresponding service scenario.

[0131] The above is a schematic scheme of another decision model training method of this embodiment. It should be noted that the technical scheme of the other decision model training method and the technical scheme of the above decision model training method belong to the same concept, and the details of the technical scheme of the other decision model training method that are not described in detail can be found in the description of the technical scheme of the above decision model training method.

[0132] The following combination Figure 5, taking the application of the decision model training method provided in this specification in a game scenario as an example, the decision model training method is further described. Figure 5 A processing flow chart of a decision model training method provided by an embodiment of the present specification is shown, which specifically includes the following steps.

[0133] Step S502: acquiring sample data associated with the target language model, and testing the sample data according to a preset decision data construction strategy.

[0134] Step S504: When it is determined according to the detection result that the sample data is of the single-step decision data type, the sample data is processed according to the decision data construction strategy.

[0135] Step S506, determining the sample state data, sample action data and sample reward data according to the processing result, and constructing the sample sequence data based on the sample state data, sample action data and sample reward data.

[0136] Step S508, extracting sample state data and sample action data from the sample sequence data and inputting them into the target language model for processing to obtain predicted reward data.

[0137] Step S510, using the sample reward data and predicted reward data in the sample sequence data, the target language model is updated to an initial decision model.

[0138] Specifically, sample reward data is extracted from the sample sequence data, and a target loss value is calculated based on the sample reward data and the predicted reward data; the target language model is adjusted according to the target loss value, and it is detected whether the adjusted target language model meets the training stop condition; if not, candidate sample data associated with the target language model is selected as sample data, and a detection step of the sample data is executed according to a preset decision data construction strategy; if so, the adjusted target language model is used as the initial decision model.

[0139] Step S512, when a game character decision task triggered by the initial decision model is detected, game data associated with the game character decision task is obtained.

[0140] Step S514, processing the game data according to the decision data construction strategy to obtain game state data, game action data and game reward data.

[0141] Step S516, constructing game sequence data based on the game state data, game action data and game reward data.

[0142] Step S518, input the game state data and game action data in the game sequence data into the initial decision model for processing to obtain predicted game reward data.

[0143] Step S520, optimizing the initial decision model based on the game reward data and the predicted game reward data in the game sequence data.

[0144] Step S522, generating a game decision model that meets the service training stop condition and matches the game character decision task according to the optimization result, wherein the game decision model is used to make action decisions for the game character in the game character decision task.

[0145] In summary, in order to improve the effectiveness and adaptability of the decision model in the decision task and facilitate downstream service applications. A general decision model can be trained first. In this process, the sample data of the associated target language model can be processed into sample sequence data according to the preset decision data construction strategy; at this time, the sample state data and sample action data can be extracted from the sample sequence data and input into the target language model for processing, so that the predicted reward data can be obtained according to the processing results; on this basis, the sample reward data and predicted reward data in the sample sequence data can be used to update the target language model to the initial decision model; the use of the adjusted language model as the decision model can make the model have stronger generalization ability. Thereafter, the service data corresponding to the service task is processed into service sequence data according to the decision data construction strategy, and the initial decision model is optimized to the service decision model corresponding to the service task using the service sequence data. It can be realized that the general decision model is trained as a service decision model for the associated service task, so that the service decision model can have stronger decision-making ability under the service task dimension, and is more adaptable to the service environment corresponding to the service task, and is more convenient for downstream service use.

[0146] Corresponding to the above method embodiment, this specification also provides a decision model training device embodiment, Figure 6 FIG. 1 is a schematic diagram showing the structure of a decision model training device provided by an embodiment of the present specification. Figure 6 As shown, the device comprises:

[0147] A strategy processing module 602 is configured to construct a strategy according to preset decision data and process the sample data associated with the target language model into sample sequence data;

[0148] The model processing module 604 is configured to extract sample state data and sample action data from the sample sequence data and input them into the target language model for processing to obtain predicted reward data;

[0149] An updating model module 606 is configured to update the target language model to an initial decision model using the sample reward data and the predicted reward data in the sample sequence data;

[0150] The optimization model module 608 is configured to process the service data corresponding to the service task into service sequence data according to the decision data construction strategy, and optimize the initial decision model into a service decision model corresponding to the service task using the service sequence data.

[0151] In an optional embodiment, the policy processing module 602 is further configured to:

[0152] Acquire sample data associated with a target language model; detect the sample data according to a preset decision data construction strategy; when it is determined according to the detection result that the sample data is a single-step decision data type, process the sample data according to the decision data construction strategy; determine sample state data, sample action data and sample reward data according to the processing result, and construct sample sequence data based on the sample state data, the sample action data and the sample reward data.

[0153] In an optional embodiment, the policy processing module 602 is further configured to:

[0154] When it is determined according to the detection result that the sample data is a sequence decision data type, the decision step information of the sample data is detected according to the decision data construction strategy; the sample data is divided into at least two sample sub-data based on the decision step information; the at least two sample sub-data are processed separately according to the decision data construction strategy, and sample sequence data is constructed according to the at least two processed sample sub-data.

[0155] In an optional embodiment, the update model module 606 is further configured to:

[0156] Extracting sample reward data from the sample sequence data, and calculating a target loss value based on the sample reward data and the predicted reward data; adjusting the parameters of the target language model according to the target loss value, and detecting whether the adjusted target language model meets the training stop condition; if not, selecting candidate sample data associated with the target language model as the sample data, and executing a step of processing the sample data associated with the target language model into sample sequence data according to a preset decision data construction strategy; if so, using the adjusted target language model as the initial decision model.

[0157] In an optional embodiment, the optimization model module 608 is further configured to:

[0158] When a service task triggered by the initial decision model is detected, service data associated with the service task is obtained; the service data is processed according to the decision data construction strategy to obtain service status data, service action data and service reward data; and service sequence data is constructed based on the service status data, the service action data and the service reward data.

[0159] In an optional embodiment, the optimization model module 608 is further configured to:

[0160] The service status data and service action data in the service sequence data are input into the initial decision model for processing to obtain predicted service reward data; the initial decision model is optimized based on the service reward data in the service sequence data and the predicted service reward data; and a service decision model that meets the service training stop condition and matches the service task is generated according to the optimization result, wherein the service decision model is used to predict service decision information for the service decision tasks associated with the service task.

[0161] In an optional embodiment, the device further comprises:

[0162] A determination module is configured to determine multiple data sources associated with the target language model; extract the sample data from the multiple data sources, and execute the step of processing the sample data associated with the target language model into sample sequence data according to a preset decision data construction strategy; wherein, after the step of optimizing the initial decision model into a service decision model corresponding to the service task using the service sequence data is executed, it also includes: obtaining a verification sample associated with the service task, and using the verification sample to verify the service decision model.

[0163] The above is a schematic scheme of a decision model training device of this embodiment. It should be noted that the technical scheme of the decision model training device and the technical scheme of the decision model training method described above belong to the same concept, and the details not described in detail in the technical scheme of the decision model training device can be found in the description of the technical scheme of the decision model training method described above.

[0164] Corresponding to the above method embodiment, this specification also provides an information processing device embodiment, Figure 7 FIG. 1 shows a schematic diagram of the structure of an information processing device provided by an embodiment of the present specification. Figure 7 As shown, the device comprises:

[0165] The information receiving module 702 is configured to receive service information submitted by the client for the target service;

[0166] The information processing module 704 is configured to input the service information into the service decision model associated with the target service for processing to obtain service decision information, wherein the service decision model is obtained by training the above method;

[0167] The information sending module 706 is configured to feed back the service decision information to the client.

[0168] The above is a schematic scheme of an information processing device of this embodiment. It should be noted that the technical scheme of the information processing device and the technical scheme of the above information processing method belong to the same concept, and the details not described in detail in the technical scheme of the information processing device can be referred to the description of the technical scheme of the above information processing method.

[0169] Corresponding to the above method embodiment, this specification also provides another decision model training device embodiment, Figure 8 FIG. 2 shows a schematic diagram of the structure of another decision model training device provided by an embodiment of the present specification. Figure 8 As shown, the device is applied to a cloud-side device, including:

[0170] The processing module 802 is configured to process the sample data associated with the target language model into sample sequence data according to a preset decision data construction strategy;

[0171] An input module 804 is configured to extract sample state data and sample action data from the sample sequence data and input them into the target language model for processing to obtain predicted reward data;

[0172] An updating module 806 is configured to update the target language model to an initial decision model using the sample reward data and the predicted reward data in the sample sequence data;

[0173] The determination module 808 is configured to determine the service data according to the optimization request submitted by the terminal device for the initial decision model when receiving the optimization request;

[0174] The optimization module 810 is configured to process the service data into service sequence data according to the decision data construction strategy, and optimize the initial decision model into a service decision model using the service sequence data.

[0175] The above is a schematic scheme of another decision model training device of this embodiment. It should be noted that the technical scheme of the another decision model training device and the technical scheme of the another decision model training method mentioned above belong to the same concept, and the details of the technical scheme of the another decision model training method that are not described in detail can all be referred to the description of the technical scheme of the another decision model training method mentioned above.

[0176] Fig. 9 The structure block diagram of a computing device 900 provided according to an embodiment of the present specification is shown. The components of the computing device 900 include but are not limited to a memory 910 and a processor 920. The processor 920 is connected to the memory 910 via a bus 930, and a database 950 is used to store data.

[0177] The computing device 900 also includes an access device 940 that enables the computing device 900 to communicate via one or more networks 960. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 940 may include one or more of any type of network interface (e.g., a network interface card (NIC)) that is wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, and a near field communication (NFC).

[0178] In one embodiment of the present specification, the above components of the computing device 900 and Fig. 9 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Fig. 9 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0179] The computing device 900 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 900 may also be a mobile or stationary server.

[0180] Among them, the processor 920 is used to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the above-mentioned decision model training method or information processing method.

[0181] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above-mentioned decision model training method or information processing method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above-mentioned decision model training method or information processing method.

[0182] An embodiment of the present specification also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the above-mentioned decision model training method or information processing method.

[0183] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned decision model training method or information processing method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above-mentioned decision model training method or information processing method.

[0184] An embodiment of the present specification also provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned decision model training method or information processing method.

[0185] The above is a schematic scheme of a computer program of this embodiment. It should be noted that the technical scheme of the computer program and the technical scheme of the above-mentioned decision model training method or information processing method belong to the same concept, and the details not described in detail in the technical scheme of the computer program can be referred to the description of the technical scheme of the above-mentioned decision model training method or information processing method.

[0186] An embodiment of the present specification also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the steps of the above-mentioned decision model training method or information processing method.

[0187] The above is a schematic scheme of a computer program product of this embodiment. It should be noted that the technical scheme of the computer program product and the technical scheme of the above-mentioned decision model training method or information processing method belong to the same concept, and the details not described in detail in the technical scheme of the computer program product can be referred to the description of the technical scheme of the above-mentioned decision model training method or information processing method.

[0188] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0189] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0190] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0191] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0192] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A decision model training method, comprising: According to a preset decision data construction strategy, the sample data associated with the target language model is processed into sample sequence data; Extracting sample state data and sample action data from the sample sequence data and inputting them into the target language model for processing to obtain predicted reward data; Using the sample reward data and the predicted reward data in the sample sequence data, updating the target language model to an initial decision model; The service data corresponding to the service task is processed into service sequence data according to the decision data construction strategy, and the initial decision model is optimized into a service decision model corresponding to the service task using the service sequence data.

2. The decision model training method according to claim 1, wherein the sample data associated with the target language model is processed into sample sequence data according to a preset decision data construction strategy, comprising: Obtain sample data associated with the target language model; Testing the sample data according to a preset decision data construction strategy; When it is determined according to the detection result that the sample data is a single-step decision data type, processing the sample data according to the decision data construction strategy; Sample state data, sample action data and sample reward data are determined according to the processing results, and sample sequence data are constructed based on the sample state data, the sample action data and the sample reward data.

3. The decision model training method according to claim 2, after the step of detecting the sample data according to the preset decision data construction strategy is executed, it also includes: When it is determined according to the detection result that the sample data is a sequence decision data type, detecting the decision step number information of the sample data according to the decision data construction strategy; Dividing the sample data into at least two sample sub-data based on the decision step number information; The at least two sample sub-data are processed respectively according to the decision data construction strategy, and sample sequence data are constructed according to the at least two processed sample sub-data.

4. The decision model training method according to claim 1, wherein the step of updating the target language model to an initial decision model by using the sample reward data and the predicted reward data in the sample sequence data comprises: Extracting sample reward data from the sample sequence data, and calculating a target loss value based on the sample reward data and the predicted reward data; Adjusting the parameters of the target language model according to the target loss value, and detecting whether the adjusted target language model meets the training stop condition; If not, selecting candidate sample data associated with the target language model as the sample data, and executing a step of processing the sample data associated with the target language model into sample sequence data according to a preset decision data construction strategy; If so, the adjusted target language model is used as the initial decision model.

5. The decision model training method according to claim 1, wherein the step of processing the service data corresponding to the service task into service sequence data according to the decision data construction strategy comprises: In case of detecting a service task triggered by the initial decision model, obtaining service data associated with the service task; Processing the service data according to the decision data construction strategy to obtain service status data, service action data, and service reward data; Service sequence data is constructed based on the service status data, the service action data, and the service reward data.

6. The decision model training method according to claim 1, wherein the step of optimizing the initial decision model into a service decision model corresponding to the service task by using the service sequence data comprises: Inputting the service status data and service action data in the service sequence data into the initial decision model for processing to obtain predicted service reward data; Optimizing the initial decision model based on the service reward data in the service sequence data and the predicted service reward data; A service decision model that meets the service training stop condition and matches the service task is generated according to the optimization result, wherein the service decision model is used to predict service decision information for the service decision task associated with the service task.

7. The decision model training method according to any one of claims 1 to 6, before the step of processing the sample data associated with the target language model into sample sequence data according to the preset decision data construction strategy is executed, it also includes: Determining multiple data sources associated with the target language model; Extracting the sample data from the multiple data sources, and executing the step of processing the sample data associated with the target language model into sample sequence data according to a preset decision data construction strategy; Wherein, after the step of optimizing the initial decision model into the service decision model corresponding to the service task by using the service sequence data is performed, the method further includes: A verification sample associated with the service task is obtained, and the service decision model is verified using the verification sample.

8. An information processing method, comprising: Receive service information submitted by the client for the target service; Inputting the service information into a service decision model associated with the target service for processing to obtain service decision information, wherein the service decision model is obtained by training the method according to any one of claims 1 to 7; The service decision information is fed back to the client.

9. A decision model training method, applied to a cloud-side device, comprising: According to a preset decision data construction strategy, the sample data associated with the target language model is processed into sample sequence data; Extracting sample state data and sample action data from the sample sequence data and inputting them into the target language model for processing to obtain predicted reward data; Using the sample reward data and the predicted reward data in the sample sequence data, updating the target language model to an initial decision model; In the case of receiving an optimization request submitted by the end-side device for the initial decision model, determining service data according to the optimization request; The service data is processed into service sequence data according to the decision data construction strategy, and the initial decision model is optimized into a service decision model using the service sequence data.

10. A decision model training device, comprising: A strategy processing module is configured to construct a strategy according to preset decision data and process sample data associated with the target language model into sample sequence data; A model processing module is configured to extract sample state data and sample action data from the sample sequence data and input them into the target language model for processing to obtain predicted reward data; An updating model module is configured to update the target language model to an initial decision model using the sample reward data and the predicted reward data in the sample sequence data; The optimization model module is configured to process the service data corresponding to the service task into service sequence data according to the decision data construction strategy, and optimize the initial decision model into a service decision model corresponding to the service task using the service sequence data.

11. An information processing device, comprising: An information receiving module is configured to receive service information submitted by a client for a target service; an information processing module, configured to input the service information into a service decision model associated with the target service for processing to obtain service decision information, wherein the service decision model is obtained by training the method according to any one of claims 1 to 7; The information sending module is configured to feed back the service decision information to the client.

12. A decision model training device, applied to a cloud-side device, comprising: A processing module is configured to process the sample data associated with the target language model into sample sequence data according to a preset decision data construction strategy; An input module is configured to extract sample state data and sample action data from the sample sequence data and input them into the target language model for processing to obtain predicted reward data; An updating module, configured to update the target language model to an initial decision model using the sample reward data and the predicted reward data in the sample sequence data; A determination module, configured to determine service data according to the optimization request submitted by the terminal device for the initial decision model when receiving the optimization request; The optimization module is configured to process the service data into service sequence data according to the decision data construction strategy, and optimize the initial decision model into a service decision model using the service sequence data.

13. A computing device comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method described in any one of claims 1 to 9 are implemented.

14. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the steps of the method according to any one of claims 1 to 9.

15. A computer program product, comprising a computer program or instructions, which implement the steps of the method according to any one of claims 1 to 9 when executed by a processor.

Images