Rule alignment method and apparatus for model, and device, medium and product

Abstract

Provided in the present disclosure are a rule alignment method and apparatus for a model, and a device, a medium and a product. The method comprises: for each rule among a plurality of rules, obtaining a plurality of sub-rules corresponding to each rule, wherein each sub-rule is used for describing one aspect among a plurality of aspects of the corresponding rule; on the basis of the plurality of rules and the plurality of sub-rules corresponding to each rule, generating a first set of training samples and a second set of training samples for a pre-trained content generation model, wherein the first set of training samples is configured to train the content generation model to determine the correspondence between the sub-rules and the rules, and the second set of training samples is configured to train the content generation model to generate content on the basis of the sub-rules; and at least on the basis of the first set of training samples and the second set of training samples, executing fine-tuning training on the content generation model, so as to obtain a trained content generation model. Thus, the accuracy of the output of a model can be improved.

Description

Methods, apparatuses, devices, media, and products for rule alignment for models TECHNICAL FIELD

[0001] Example embodiments of the present disclosure generally relate to the field of computers, and in particular, to methods, apparatuses, electronic devices, computer-readable storage media, and computer program products for rule alignment for models. BACKGROUND

[0002] With the development of machine learning techniques, it has been possible to utilize trained machine learning models to perform tasks in a variety of application environments. Machine learning models can be trained with different training samples, and the machine learning models trained with different training samples can be used to perform different tasks. Machine learning models can have a plurality of hierarchical structures. Trained machine learning models can have a large number of parameters distributed in different hierarchical structures of the machine learning models to represent and understand subtle interrelationships of model inputs. For example, a machine learning model can understand the synthetic structure between phonemes of speech, word order of text, pixels of images, etc. through weight parameters of neural network layers. The accuracy of the parameters affects the ability of the machine learning model, and the training quality of the machine learning model affects the accuracy of the parameters of the machine learning model.

[0003] SUMMARY

[0004] In a first aspect of the present disclosure, a method for rule alignment for models is provided. The method comprises: obtaining, for each rule in a plurality of rules, a plurality of sub-rules corresponding to each rule, each sub-rule being used to describe one aspect of a plurality of aspects of the corresponding rule; generating, based on the plurality of rules and the plurality of sub-rules corresponding to each rule, a first set of training samples and a second set of training samples for a pre-trained content generation model, wherein the first set of training samples is configured to train the content generation model to determine a correspondence between the sub-rules and the rules, and the second set of training samples is configured to train the content generation model to generate content based on the sub-rules; and performing, based on at least the first set of training samples and the second set of training samples, fine-tuning training on the content generation model to obtain a trained content generation model, wherein the trained content generation model is configured to generate content by following the plurality of rules based on a prompt input.

[0005] In a second aspect of the disclosure, an apparatus for rule alignment of a model is provided. The apparatus includes: a sub-rule obtaining module configured to, for each rule in a plurality of rules, obtain a plurality of sub-rules corresponding to each rule, each sub-rule being used to describe one aspect in a plurality of aspects of the corresponding rule; a sample generating module configured to generate, based on the plurality of rules and the plurality of sub-rules corresponding to each rule, a first set of training samples and a second set of training samples for a pre-trained content generation model, wherein the first set of training samples are configured to train the content generation model to determine a correspondence between the sub-rules and the rules, and the second set of training samples are configured to train the content generation model to generate content based on the sub-rules; and a model training module configured to perform fine-tuning training on the content generation model based on at least the first set of training samples and the second set of training samples to obtain a trained content generation model, wherein the trained content generation model is configured to generate content by following the plurality of rules based on a prompt word input.

[0006] In a third aspect of the disclosure, an electronic device is provided. The electronic device includes: at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, cause the electronic device to perform the method according to the first aspect of the disclosure.

[0007] In a fourth aspect of the disclosure, a computer-readable storage medium is provided, having stored thereon a computer program which, when executed by a processor, causes the processor to perform the method according to the first aspect of the disclosure.

[0008] In a fifth aspect of the disclosure, a computer program product is provided. The computer program product is tangibly stored in a computer storage medium and includes computer- executable instructions that, when executed by a device, cause the device to perform the method of the first aspect.

[0009] It should be understood that the content described in this section is not intended to limit the key features or important features of the embodiments of the disclosure, nor to limit the scope of the disclosure. Other features of the disclosure will become apparent through the following description. BRIEF DESCRIPTION OF DRAWINGS

[0010] The above and other features, advantages and aspects of embodiments of the present disclosure will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings in which:

[0011] FIG. 1 shows a schematic diagram of an example environment in which embodiments according to the present disclosure can be implemented;

[0012] FIGS. 2A-2C illustrate examples of rule alignment for a model;

[0013] FIG. 3 illustrates a flowchart of a process for rule alignment for a model, in accordance with some embodiments of the present disclosure;

[0014] FIG. 4 illustrates an example of rule partitioning, in accordance with some embodiments of the present disclosure;

[0015] FIG. 5 illustrates an example of sample evaluation, in accordance with some embodiments of the present disclosure;

[0016] FIG. 6 illustrates a schematic structural block diagram of an example apparatus for rule alignment for a model, in accordance with some embodiments of the present disclosure; and

[0017] FIG. 7 illustrates a block diagram of an electronic device that can implement one or more embodiments of the present disclosure. DETAILED DESCRIPTION

[0018] Embodiments of the present disclosure will be described below in greater detail with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be interpreted as being limited to the embodiments set forth herein; rather, these embodiments are provided so as to more completely and thoroughly understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for exemplary purposes only and are not intended to limit the scope of protection of the present disclosure.

[0019] It is noted that the headings provided herein are not limitations of the disclosure. Various embodiments are described throughout this document and any type of embodiment can be included under any heading. Additionally, embodiments described in any heading can be combined with any other embodiment described in the same heading and / or a different heading in any manner.

[0020] In the description of embodiments of the present disclosure, the term "includes" and its derivatives are to be interpreted as including without limitation. The term "based on" is to be interpreted as "based, at least in part, on." The term "one embodiment" or "an embodiment" are to be interpreted as "at least one embodiment." The term "some embodiments" are to be interpreted as "at least some embodiments." Other explicit and implicit definitions can also be included below. The terms "first," "second," etc. can refer to different or same objects. Other explicit and implicit definitions can also be included below.

[0021] The embodiments of the present disclosure can involve data of users, acquisition and / or use of data, etc. These aspects all comply with the corresponding laws and regulations and relevant provisions. In the embodiments of the present disclosure, all data collection, acquisition, processing, processing, forwarding, use, etc. are carried out on the premise that the user is aware of and confirms. Accordingly, when implementing the embodiments of the present disclosure, the type of data or information that can be involved, the use range, the use scenario, etc. should be notified to the user and the authorization of the user should be obtained through appropriate means according to the relevant laws and regulations. The specific notification and / or authorization mode can vary according to the actual situation and application scenario, and the scope of the present disclosure is not limited in this respect.

[0022] In the present specification and embodiments, if the scheme involves processing of personal information, the processing will be carried out on the premise of having a legal basis (for example, obtaining the consent of the subject of personal information, or being necessary for the performance of a contract, etc.), and will be carried out only within the prescribed or agreed range. The user's refusal to process personal information other than the necessary information required for the basic function will not affect the user's use of the basic function.

[0023] As used herein, the term "model" can learn an association between a corresponding input and an output from training data, so that after training is completed, a corresponding output can be generated for a given input. The generation of a model can be based on machine learning techniques. Deep learning is a machine learning algorithm that processes input and provides a corresponding output by using multiple layers of processing units. A neural network model is one example of a model based on deep learning. In this document, a "model" can also be referred to as a "machine learning model", a "learning model", a "machine learning network", or a "learning network", which terms are used interchangeably herein.

[0024] A "neural network" is a machine learning network based on deep learning. A neural network is capable of processing input and providing a corresponding output, which typically includes an input layer and an output layer and one or more hidden layers between the input layer and the output layer. Neural networks used in deep learning applications typically include many hidden layers, thereby increasing the depth of the network. The layers of a neural network are connected in sequence, so that the output of a previous layer is provided as input to a subsequent layer, with the input layer receiving the input to the neural network and the output of the output layer as the final output of the neural network. Each layer of a neural network includes one or more nodes (also referred to as processing nodes or neurons), each of which processes input from the previous layer.

[0025] Generally, machine learning can include three stages, namely a training stage, a testing stage, and an application stage (also referred to as an inference stage). In the training stage, a given model can be trained using a large amount of training data, iteratively updating parameter values until the model is able to obtain consistent inferences from the training data that satisfy an expected objective. Through training, the model can be considered to have learned the association (also referred to as the mapping) between the input and the output from the training data. The parameter values of the trained model are determined. In the testing stage, test inputs are applied to the trained model to test whether the model is able to provide correct outputs, thereby determining the performance of the model. In the application stage, the model can be used to process actual inputs based on the parameter values obtained through training to determine the corresponding outputs.

[0026] As mentioned above, machine learning models can be utilized to perform tasks in a variety of application environments, the parameters of which affect the capabilities of the machine learning models. It can be appreciated that the larger the amount of parameters, the larger the model size of the machine learning model, and the stronger the model capabilities of the machine learning model. For machine learning models with a large amount of parameters, the training thereof requires a large amount of computing resources and power, which can bring about corresponding economic and environmental pressures. Secondly, the size and complexity of the model also increase the difficulty of adjusting and optimizing the parameters of the model. Furthermore, machine learning models with a large amount of parameters can be more sensitive to noise and anomalies in input data, which can affect the accuracy of their predictions and outputs.

[0027] In particular, for tasks associated with problems of closed rule spaces, the accuracy of the predictions and outputs of machine learning models with a large amount of parameters is poor. A closed rule space refers to a system or environment that is closed under certain conditions or rules. In this environment or system, all possible states or behaviors follow a set of predefined rules (Rules) that determine the boundaries and characteristics of the space. Any action or state transition in this space must satisfy these rules. In other words, this system or environment is “closed” because its behavior and state are constrained by a specific set of rules.

[0028] For example, a video classification problem can be defined as a closed space problem. The dimension of the rule space, i.e., the features that affect the video classification, can be determined. These features can include information at the visual level, such as object detection, person recognition, and scene analysis; information at the audio level, such as music type, frequency analysis, speech recognition, and sentiment analysis; and metadata, such as publishing time, duration, like ratio, dislike ratio, and the like. Then, in this closed rule space system, a series of rules can be defined to divide the feature space into different regions, and different regions can represent different industry categories. For example, videos in the "education" category usually have clear explanations, blackboard writing, or document display; videos in the "fitness" category usually have fitness action demonstrations and related music; and videos in the "food" category usually have food making processes and finished product displays.

[0029] It is desirable to improve the processing capability of a machine learning model for a task associated with a closed rule space problem, and to improve the accuracy of the output of the model.

[0030] Embodiments of the present disclosure propose an improved solution for rule alignment of a model. The solution includes: for each rule in a plurality of rules, obtaining a plurality of sub-rules corresponding to each rule, each sub-rule being used to describe one aspect of a plurality of aspects of the corresponding rule. Based on the plurality of rules and the plurality of sub-rules corresponding to each rule, generating a first set of training samples and a second set of training samples for a pre-trained content generation model, wherein the first set of training samples is configured to train the content generation model to determine a correspondence between the sub-rules and the rules, and the second set of training samples is configured to train the content generation model to generate content based on the sub-rules. At least based on the first set of training samples and the second set of training samples, performing fine-tuning training on the content generation model to obtain a trained content generation model, wherein the trained content generation model is configured to generate content by following the plurality of rules based on a prompt input.

[0031] In this way, embodiments of the present disclosure can help the model to align with the rules, and help to improve the processing capability of the model for a task associated with a closed rule space problem, and to improve the accuracy of the output of the model.

[0032] Various example implementations of the solution are described in further detail below in conjunction with the accompanying drawings.

[0033] FIG. 1 shows a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented. As shown in FIG. 1, the environment 100 can include an electronic device 110.

[0034] The electronic device 110 can obtain target data 102 to be executed for a target task, and determine a target processing result 112 of the target task for the target data 102. The target task here can be any appropriate task, and the corresponding target processing result 112 can be any appropriate processing result matching the target task. The target data 102 can be any appropriate type of data, including but not limited to text, audio, image, video, document, connection, code, etc. As an example, if the target task is an object recognition task and the target data 102 is an image, the electronic device 110 can determine the target processing result 112 of the object recognition task for the image, which can indicate the object recognition result of at least one object in the image.

[0035] The electronic device 110 can execute the target task on the target data 102 to determine the target processing result 112, for example, by using the trained model 120. The model 120 can be deployed on the electronic device 110, or on other devices (e.g., in a remote device). If the model 120 is deployed on other devices, the electronic device 110 can invoke the model 120 to process the target data 102 via a communication connection with the other devices.

[0036] The model 120 can be any appropriate machine learning model, which can be based on any appropriate model structure, including but not limited to a Transformer model, a convolutional neural network (CNN), a recurrent neural network (RNN), a deep neural network (DNN), etc. In some embodiments, the model 120 can also be based on a language model (LM). A language model can have the ability of question and answer by learning from a large amount of corpus. The model 120 based on the language model can have the ability of content generation. By inputting prompt word information to the model 120, the model 120 can be guided to generate content meeting the requirements of the prompt word information. Such a model is also called a generative model or a content generation model.

[0037] The electronic device 110 can include any computing system with computing capability, such as various computing devices / systems, end devices, server devices, and the like. The end device can be any type of mobile terminal, fixed terminal, or portable terminal including a mobile handset, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a media computer, a multimedia tablet, a palmtop computer, a portable gaming terminal, a VR / AR device, a Personal Communication System (PCS) device, a personal navigation device, a Personal Digital Assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an electronic book device, a game device, or any combination thereof, including accessories and peripherals of these devices, or any combination thereof.

[0038] The server device can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks, and basic cloud computing services such as big data and artificial intelligence platforms. The server device may, for example, include computing systems / servers such as mainframes, edge computing nodes, computing devices in a cloud environment, and the like.

[0039] It should be understood that the structure and function of the various elements in the environment 100 are described for illustrative purposes only, without implying any limitation on the scope of the present disclosure.

[0040] As mentioned previously, machine learning can include three stages of a training stage, a testing stage, and an application stage. In some embodiments, the training stage can include a pre-training stage and a fine-tuning stage, the testing stage can be located after the pre-training stage or the fine-tuning stage is completed, and the application stage can be located after the testing stage is completed. The electronic devices corresponding to the training stage, the testing stage, and the application stage (i.e., the electronic device for training the model, the electronic device for testing the model, and the electronic device for applying the model) can be different electronic devices or the same electronic device. It can be understood that the electronic device for training the model can include multiple electronic devices, such as an electronic device for pre-training the model and an electronic device for fine-tuning the model.

[0041] An electronic device for pre-training a model can be deployed with a model pre-training system, an electronic device for fine-tuning a model can be deployed with a model fine-tuning system, and an electronic device for applying a model can be deployed with a model application system. In a pre-training phase, the model pre-training system is used to pre-train a machine learning model (e.g., model 120) that can be configured to learn an accurate representation of data (also referred to as a feature representation of data) from training data. Before pre-training, the parameter values of the model 120 can have initial values. The pre-training of the model 120 is performed with the training data. The parameter values of the model 120 can be updated and adjusted during the pre-training. The parameter values of the model 120 that have completed pre-training have been updated and have pre-trained parameter values.

[0042] In the pre-training phase, the model 120 can learn strong generalization capability from a large amount of training data. The pre-trained model 120 can be provided to the model fine-tuning system for fine-tuning for different downstream tasks. In some implementations, for different downstream tasks, the pre-trained model 120 can be connected to different task-specific layers to build different downstream task models. This is because different outputs are needed for different downstream tasks. The pre-trained model 120 can perform extraction of feature representations for inputs and provide to the task-specific layers to provide outputs for the corresponding tasks.

[0043] In the fine-tuning phase, according to the needs of specific downstream tasks, corresponding training data can be selected to fine-tune the built downstream task models respectively. The model training algorithm is also used to update and adjust the parameters of the overall model during fine-tuning. Since the pre-trained model 120 has learned a lot of knowledge from the training data in the pre-training phase, a small amount of training data can be used in the fine-tuning phase to obtain a desired downstream task model.

[0044] In some implementations, in the pre-training phase, one or more task-specific layers can have been built for pre-training the model 120 for multiple downstream tasks according to the needs of the pre-training target. In this case, if the task-specific layers used in the downstream tasks are the same as the task-specific layers built in the pre-training, the pre-trained model 120 and the task-specific layers can be used directly to form the corresponding downstream task model. In this case, the downstream task model can not need to be fine-tuned or only need to be fine-tuned with a small amount of training data.

[0045] In the application phase, the obtained downstream task models can be provided to one or more model application systems for use. In the application phase, each downstream task model can be used to process corresponding inputs in actual scenarios and provide corresponding outputs.

[0046] FIGS. 2A-2C illustrate example 200A-200C for rule alignment of a model. The examples 200A-200C can be implemented, for example, at the electronic device 110. The examples 200A-200C are described below with reference to FIG. 1. The examples 200A-200C illustrate various example schemes for rule alignment of a model.

[0047] In example 200A, the electronic device 110 can determine a prompt input 214 based on various rules 212 within a rule space, which can be provided as part of the prompt input 214. The prompt input 214 is provided to the trained model 120. The model response 216 output by the trained model 120 is also the model output for the prompt input 214. In some embodiments, the prompt input 214 can include the target data 102, and the model response 216 can indicate the target processing result 112 of performing the target task on the target data 102.

[0048] The scheme illustrated in example 200A can directly reuse the trained model 120 without fine-tuning the model 120, which can avoid resource consumption required for model training. In addition, providing the rules to the model 120 in the form of prompt input can facilitate the model 120 to learn the rules and can improve the efficiency of rule alignment. However, the length of the input that the model 120 can receive is limited, and for scenarios with a large number of rules, it can not be possible to provide all the rules to the model 120. This can affect the model 120’s learning of all the rules and further affect the processing capability of the model 120. In addition, for scenarios with complex rules, examples matching the rules need to be provided to the model 120 together with the rules. If the examples are provided to the model 120 in the form of prompt input, this can further increase the length of the prompt input. If the examples are not provided, this can make it difficult for the model 120 to accurately understand the rules.

[0049] In example 200B, the electronic device 110 can obtain an annotated training dataset 222, which can be a training dataset formed by annotating data of various rules 212 within a rule space. The electronic device 110 can fine-tune the pre-trained model 120 using the training dataset 222 to improve the model 120’s effectiveness in identifying and classifying rules in the space. During the process of fine-tuning the trained model 120, the model response 224 output by the model 120 can indicate a processing result of processing data in the training dataset 222.

[0050] The scheme shown in example 200B is to fine-tune the pre-trained model 120 using rules, which can be applied to any appropriate rule space, such as a simple rule space, a complex rule space, a cascading rule space, etc., and has a wide range of application scenarios. Each rule space can improve the effect of the model 120 in the rule space through model fine-tuning. However, the scheme shown in example 200B requires fine-tuning the model 120, which requires additional resources such as GPU cards, power, etc. In addition, the rules in the rule space need to be split, and the quality of the training data set used for fine-tuning is very high, and the data matching requirement is also high. Each rule needs to collect or organize the training set corresponding to each sub-rule, otherwise it will cause the model 120 to have insufficient coverage of the rule space, resulting in performance loss.

[0051] In example 200C, the knowledge 232 associated with all rules 212 in the rule space can be used to train the model 120 from scratch as part of the model pre-training corpus (i.e., pre-training data). That is, the model 120 can be pre-trained in combination with the knowledge 232. This can enable the trained model 120 to have the ability to master these rules. During the pre-training process of the model 120, the model response 234 output by the model 120 can indicate the processing result of processing the pre-training data. It can be understood that the electronic device 110 can also continue to fine-tune the pre-trained model 120 subsequently.

[0052] The scheme shown in example 200C can enable the model 120 to have the ability to solve the rule space problem from the source, and the pre-trained model 120 only needs to be prompted or simply fine-tuned to better solve the rule space problem. However, such a scheme consumes a lot of resources, especially in the case of too many closed rule spaces. It is not possible to retrain the model 120 every time, which will cause a large amount of resource consumption and rapid cost increase. In addition, many problems will be encountered during the training of the model, and the entire problem solving period becomes very long, which will reduce the efficiency of the model application.

[0053] For the scheme shown in example 200B, traditionally, existing business data is often collected and then fine-tuned using these business data. However, business data often only aggregates on some rules and is less or not distributed on other rules, showing a long-tail effect, which causes the model fine-tuned directly using such data to have a deviation phenomenon. That is, underfitting occurs on some rules, overfitting occurs on some rules, and no fitting occurs on the remaining rules. Such a model will have a discount in solving the closed rule space problem. Moreover, simply using business data cannot enable the model to master all rules in the space, but only rules can be learned through data, and the specific rules and boundaries cannot be intuitively learned. This will cause greater model performance loss in the problem solving process.

[0054] Therefore, based on the fine-tuning scheme shown in example 200B, the model alignment rule space can be achieved by rule space splitting, data task construction, and model fine-tuning, so that the related problems of the rule space can be efficiently and accurately solved. In addition, through rule splitting, task construction, and data quality evaluation, etc., the present disclosure can be extended to any problem of reusing models to align a certain closed rule space. Some example processes of the present disclosure will be described in detail below with reference to FIG. 3.

[0055] FIG. 3 shows a flowchart of a process 300 for rule alignment of a model according to some embodiments of the present disclosure. The process 300 can be implemented at the electronic device 110. The process 300 will be described below with reference to FIG. 1.

[0056] As shown in FIG. 3, at block 310, the electronic device 110 obtains, for each rule in a plurality of rules, a plurality of sub-rules corresponding to each rule, each sub-rule being used to describe one aspect of a plurality of aspects of the corresponding rule.

[0057] The closed rule space can include a plurality of rules, and the definition of these rules is usually unstructured, that is, each rule is usually defined by theorems, texts, and formulas. The plurality of rules contains overlapping problems, boundary problems, and exemption cases. In embodiments of the present disclosure, to improve the effect of model fine-tuning, the plurality of rules in the closed rule space can be processed to make the plurality of rules more granular and more bounded. Specifically, each rule can be split according to a predetermined splitting principle or method.

[0058] The splitting process can be performed by an electronic device (which can be the electronic device 110 or other electronic devices) or manually. That is, the plurality of sub-rules corresponding to each rule can be obtained by splitting the rule based on a predetermined splitting principle or method by an electronic device, or by splitting the rule based on a predetermined splitting principle or method by a specific user, which is not limited by the present disclosure. The rule can be split to the smallest granularity to obtain the plurality of sub-rules. For example, rule A can cover 10 aspects, and rule A can include 10 sub-rules, each sub-rule corresponding to one aspect.

[0059] The predetermined rule splitting principle or method can indicate, for example, that each rule is split into a plurality of provisions that are independent of each other, i.e., there is no overlap between different provisions. The predetermined rule splitting principle or method can also indicate, for example, that each provision is understandable by common sense. The predetermined rule splitting principle or method can also indicate, for example, that each provision should contain a complete range. The predetermined rule splitting principle or method can also indicate, for example, that the range of the plurality of provisions corresponding to each rule covers all the space of the rule as much as possible. The splitting principle or method can be adjusted according to needs and business, and can be two-level splitting, multi-level splitting, or even tree structure.

[0060] Referring to FIG. 4, FIG. 4 shows an example 400 of rule division according to some embodiments of the present disclosure. As shown in FIG. 4, the closed rule space contains three rules (e.g., rule-0, rule-1, and rule-2) in total, and each rule can be split into a plurality of provisions. For example, rule-0 can be split to obtain 16 provisions (e.g., P0 to P15 in the space of rule-0 in the figure), rule-1 can be split to obtain 16 provisions (e.g., P0 to P15 in the space of rule-1 in the figure), and rule-2 can be split to obtain 9 provisions (e.g., P0 to P8 in the space of rule-2 in the figure). It should be noted that even if represented as the same symbol, the provisions belonging to different rules are different. For example, the provision P0 in rule-0, the provision P0 in rule-1, and the provision P0 in rule-2 are three different provisions.

[0061] Referring to the splitting principle or method mentioned above, it can be found that the plurality of provisions after splitting of rule-0 and rule-1 fill the rule space as much as possible, indicating that this is a better split, but rule-2 only covers part of it, indicating that some provisions are missing, and such a split can not be ideal. In addition, if there is a large overlap between two provisions, it indicates that the two provisions are not independent of each other, and it can be possible to combine them into one provision.

[0062] Thus, by splitting the rules, the abstract and unstructured rules can be converted into provisions that can be understood based on common sense, which helps the understanding and learning of the model itself.

[0063] After splitting the abstract rules into sub-rules, training data also needs to be constructed to convert the rule-related knowledge into fine-tuning training sets, so as to help the model learn the meaning, definition and boundary of each rule in the fine-tuning process. Each training sample in the fine-tuning training set includes a prompt word input and a model response, i.e. a pair <prompt, response>, for performing supervised training on the model. Specifically, at block 320, the electronic device 110 generates a first set of training samples and a second set of training samples for the pre-trained content generation model (e.g. the model 120) based on the plurality of rules and the plurality of sub-rules corresponding to each rule.

[0064] In embodiments of the present disclosure, on the basis of the pre-trained content generation model, it is expected to further fine-tune the content generation model through supervised learning by constructing a small amount of labeled training samples, so as to train the model to align to a specific rule space, so as to be able to follow the expected rules to complete the content generation process. The content generation model can generate a model response that meets the requirements of the prompt word input under the guidance of the prompt word input. Depending on the specific configuration of the content generation model, the model response can include data of various modalities. In some embodiments, the content generation model to be fine-tuned can be configured as a text generation model (e.g. Text2Text based on text generation, Image2Text based on image generation, etc.) for generating text, an image generation model (e.g. Text2Image based on text generation, etc.) for generating images, a video generation model for generating videos, an audio generation model for generating audio, and any other appropriate type of model.

[0065] The first set of training samples is configured to train the content generation model to determine the correspondence between the sub-rules and the rules. The first set of training samples is constructed for training the content generation model to be able to implement the sub-rule attribution and inclusion relationship task. In some embodiments, each training sample in the first set of training samples includes a first sample prompt word input and a corresponding first model response. Each first sample prompt word input may, for example, include a question about the sub-rules included in the rule, and the corresponding first model response may, for example, include a plurality of sub-rules included in the questioned rule. As an example, the first sample prompt word input can include the text "which sub-rules are included in rule A", and the corresponding first model response can include the text "rule A is about …, which includes the following sub-rules: …". The first set of training samples can be automatically constructed through the attribution and inclusion relationship between the rules and the sub-rules.

[0066] The second set of training samples is configured to train the content generation model to generate content based on the guidelines. The second set of training samples is constructed for training the content generation model to achieve the task of generating content based on the guidelines. In some embodiments, each training sample in the second set of training samples includes a second sample prompt input and a corresponding second model response. Each second sample prompt input may, for example, be configured to direct the content generation model to generate content based on a given guideline, and the corresponding second model response may, for example, include sample content generated based on the given guideline. The content here can be any appropriate type of content such as text, audio, image, video, etc., and the specific type of text can be determined based on the content generation model to be fine-tuned. As an example, if the content generation model is a text generation model, the content to be directed the content generation model to generate is also text, in which case the second sample prompt input can include the text “generate some text about guideline A”, and the corresponding first model response can include the text “the text about guideline A is as follows:...”. In some embodiments, the second set of training samples, particularly the model responses in the second set of training samples, can also be generated with the aid of trained machine learning models. Depending on the content to be generated, which can be text, image, video, or other, a corresponding trained machine learning model can be selected to help generate the model responses.

[0067] In some embodiments, the electronic device 110 can further generate, based on the plurality of guidelines and the plurality of rules corresponding to each guideline, at least one other set of training samples for the content generation model.

[0068] In some embodiments, the electronic device 110 can further generate a third set of training samples for the content generation model based on the plurality of rules and the plurality of sub-rules corresponding to each rule. The third set of training samples are configured to train the content generation model to determine the definition of the sub-rules contained in the rules and to generate examples associated with the sub-rules. The third set of training samples are constructed for training the content generation model to be able to implement the sub-rule parsing and example analysis tasks. The third set of training samples may, for example, include third sample prompt inputs and corresponding third model responses. Each third sample prompt input may, for example, be configured to direct the content generation model to generate the definition of the sub-rules contained in the rules, and the corresponding third model response may, for example, include the generated definition and sample examples. The sample examples and definitions here can be pre-generated, which can be obtained for generating the third set of training samples. As an example, the third sample prompt input can include the text “How to understand rule A”, and the corresponding third model response can include the text “Rule A involves … aspects, which can include the following examples: …”. In some embodiments, the third set of training samples can also be generated with the aid of a trained machine learning model, in particular, to generate the model responses in the third set of training samples, including the examples therein. Depending on the content to be generated, which can be text, image, video, or other, a corresponding trained machine learning model can be selected to help generate the model responses.

[0069] In some embodiments, the electronic device 110 can further generate a fourth set of training samples for the content generation model based on the plurality of rules and the plurality of sub-rules corresponding to each rule. The fourth set of training samples may, for example, include fourth sample prompt inputs and corresponding fourth model responses. Each fourth prompt input may, for example, be configured to direct the content generation model to generate domain knowledge associated with the rules, and the corresponding fourth model response may, for example, include the associated domain knowledge. The domain knowledge here can be pre-generated, which can be obtained for generating the fourth set of training samples. For example, the domain knowledge can be obtained from web links. The fourth set of training samples are constructed for training the content generation model to be able to learn the rule-related knowledge. For example, if the rules are rules in the field of advertising, advertising-related knowledge can be collected and added to the training samples. The fourth sample prompt input can include the text “Please explain the brand safety product in the advertising process”, and the corresponding fourth model response can include the text “Brand safety is a product designed to protect advertisers’ ads in a safe environment …”.

[0070] Through the construction of the training samples of the above tasks, in the process of fine-tuning, the content generation model can better master most of the knowledge related to the rules, thereby ensuring that the model truly understands each rule and sub-rule, which provides a foundation guarantee for solving problems related thereto.

[0071] It can be understood that the multiple sets of training samples can further include more sets of training samples, such as a fifth set of training samples, a sixth set of training samples, and the like, and the present disclosure does not limit the same. The number of training samples included in the multiple sets of training samples can be determined based on the task to be performed by the content generation model.

[0072] In some embodiments, the electronic device 110 can further evaluate the generated multiple sets of training samples (at least including the first set of training samples and the second set of training samples) to determine the quality of the multiple sets of training samples. The electronic device 110 may, for example, determine at least one sample evaluation indicator for each of the plurality of rules based on the second set of training samples, and update the second set of training samples and / or update at least one rule of the plurality of rules corresponding to the sub-rules based on the at least one sample evaluation indicator for each of the plurality of rules.

[0073] If the multiple sample evaluation indicators are included, the multiple sample evaluation indicators can correspond to multiple evaluation dimensions. For example, the multiple sample evaluation indicators can include a sample evaluation indicator of a diversity dimension and a sample evaluation indicator of a representativeness dimension. The sample evaluation indicator of the diversity dimension can also be referred to as a first sample evaluation indicator, which can be used to indicate the sample diversity of the plurality of sub-rules corresponding to the rule. In short, the higher the value corresponding to the first sample evaluation indicator, the higher the spatial coverage of the content corresponding to the plurality of sub-rules to the corresponding rule (i.e., the more comprehensive the content corresponding to the plurality of sub-rules). The sample evaluation indicator of the representativeness dimension can also be referred to as a second sample evaluation indicator, which can be used to indicate the sample representativeness corresponding to the rule.

[0074] Regarding the specific way of determining the first sample evaluation indicator, in some embodiments, for a given rule of the plurality of rules, the electronic device 110 can determine the total number of samples associated with the plurality of sub-rules corresponding to the given rule and the number of samples associated with each of the plurality of sub-rules from the second set of training samples related to the task of generating content based on the sub-rules. The electronic device 110 can determine the first sample evaluation indicator of the plurality of sub-rules corresponding to the given rule based on the comparison between the number of samples associated with each of the plurality of sub-rules and the total number of samples.

[0075] The electronic device 110 may, for example, determine the first sample evaluation indicator based on the following formula:

[0076] where the numerator is the area of the rule space covered by the jth sub-rule training sample, which can be represented by the number of training samples associated with the jth sub-rule. Δs i,jDiversity represents the spatial area of the ith training sample in the jth rule in the plurality of rules, which can indicate the number of samples in the jth rule. In equation (1), the denominator is the total area of the entire closed rule space (all rules) corresponding to the given rule, which can be represented by the total number of training samples associated with the given rule. That is, for each rule, the number of data samples in the rule can be calculated, and this number is divided by the number of data samples in all rules to determine the Diversity. The greater the value of Diversity, the higher the diversity, and the smaller the value, the lower the diversity. When constructing the training set for model fine-tuning, it is desirable to construct as diverse data as possible, so the value of Diversity is expected to be as high as possible.

[0077] Regarding the specific way of determining the second sample evaluation indicator, in some embodiments, for a given rule in the plurality of rules, the electronic device 110 can determine, from the second set of training samples, the minimum number of samples and the maximum number of samples associated with each of the plurality of rules corresponding to the given rule. The electronic device 110 can determine the second sample evaluation indicator of the given rule based on a comparison between the minimum number of samples and the maximum number of samples.

[0078] The electronic device 110 may, for example, determine the second sample evaluation indicator of the given rule based on the following formula:

[0079] where Δs j Diversity represents the spatial area of the ith training sample in the jth rule in the plurality of rules, which can indicate the number of samples in the jth rule. In equation (1), the denominator is the total area of the entire closed rule space (all rules) corresponding to the given rule, which can be represented by the total number of training samples associated with the given rule. That is, for each rule, the number of data samples in the rule can be calculated, and this number is divided by the number of data samples in all rules to determine the Diversity. The greater the value of Diversity, the higher the diversity, and the smaller the value, the lower the diversity. When constructing the training set for model fine-tuning, it is desirable to construct as diverse data as possible, so the value of Diversity is expected to be as high as possible.

[0080] Referring to FIG. 5, FIG. 5 shows an example 500 of sample evaluation according to some embodiments of the present disclosure. As shown in FIG. 5, rule-0 includes 16 sub-rules P0 to P15, each small circle represents a space of a sub-rule, and the pentagram represents a training sample in the corresponding sub-rule. For sub-rule P0, the training samples cover the entire sub-rule space as much as possible, which indicates that the diversity and representativeness of the training samples corresponding to sub-rule P0 are both good. For sub-rule P1, there are many training samples, and they are representative of the space, but there are many similar training samples, so the diversity is insufficient. For sub-rule P3, the data distribution is very diversified, but these training samples not only cover the sub-rule space but also cover other spaces, which indicates that the boundary of the training data is not clear, there is diversity, but not representativeness.

[0081] In some embodiments, the electronic device 110 can also determine whether the quality of each set of training samples meets a quality condition based on the evaluation results of the sets of training samples (i.e., the respective at least one sample evaluation indicator). For each set of training samples, the electronic device 110 may, for example, determine that the quality of the set of training samples meets the quality condition in response to each of the at least one sample evaluation indicator corresponding to the set of training samples reaching the corresponding threshold value.

[0082] In some embodiments, to improve the quality of each set of training samples, the electronic device 110 can also label each sub-rule and de-duplicate the plurality of sub-rules and the training samples of the plurality of sub-rules. In some embodiments, if there are many training samples corresponding to a sub-rule, the electronic device 110 can also sample a plurality of training samples based on a predetermined number. In some embodiments, if there are few training samples corresponding to a sub-rule, the electronic device 110 can also obtain more training samples for the sub-rule (e.g., by means of artificial or by means of a model) on the basis of ensuring diversity and representativeness, and supplement the obtained new training samples to the existing sets of training samples.

[0083] After the training samples are constructed, at block 330, the electronic device 110 performs fine-tuning training on the content generation model based at least on the first set of training samples and the second set of training samples to obtain a trained content generation model, where the trained content generation model is configured to generate content by following the plurality of rules based on the prompt input.

[0084] The electronic device 110 can merge the multiple sets of training samples (e.g., which can further include the third set of training samples and / or the fourth set of training samples) including at least the first set of training samples and the second set of training samples to determine a training dataset for fine-tuning the content generation model. In some embodiments, the multiple sets of training samples can have a certain order in the training dataset, and the electronic device 110 may, for example, fine-tune the content generation model with the respective set of training samples in the order. The order of the multiple sets of training samples may, for example, be the third set of training samples - the first set of training samples - the second set of training samples - the fourth set of training samples. The electronic device 110 can perform fine-tuning training on the content generation model with this training dataset to align the content generation model with the closed rule space. The fine-tuned content generation model can master the knowledge related to the rule space and can better solve the related problems in the closed rule space.

[0085] Thus, splitting the abstract unstructured rules into rules that can be understood by common sense can help the data annotators reduce the learning cost and make the model fine-tuning easier. By constructing the tasks related to the closed rule space, the model can effectively learn the related knowledge and fundamentally master these rules. By defining the quality indicators of the data rule annotation data, the quality of each data sample can be described, thereby helping to improve the performance of the prediction model. Through data construction and fine-tuning, the model can be effectively aligned with a certain closed rule space, thereby accurately solving the related problems in the space. In summary, the model can be aligned with the rules, which helps to improve the processing ability of the model for tasks related to the closed rule space and improve the accuracy of the output of the model.

[0086] Embodiments of the present disclosure also provide a corresponding apparatus for implementing the above method or process. FIG. 6 shows a schematic structural block diagram of an example apparatus 600 for rule alignment of a model according to certain embodiments of the present disclosure. The apparatus 600 can be implemented as or included in the electronic device 110. The various modules / components in the apparatus 600 can be implemented by hardware, software, firmware, or any combination thereof.

[0087] As shown in FIG. 6, the apparatus 600 includes a rule obtaining module 610 configured to, for each rule in a plurality of rules, obtain a plurality of sub-rules corresponding to the rule, each sub-rule being used to describe one aspect in a plurality of aspects of the corresponding rule; a sample generating module 620 configured to generate, based on the plurality of rules and the plurality of sub-rules corresponding to each rule, a first set of training samples and a second set of training samples for a pre-trained content generation model, wherein the first set of training samples are configured to train the content generation model to determine a correspondence between a sub-rule and a rule, and the second set of training samples are configured to train the content generation model to generate content based on a sub-rule; and a model training module 630 configured to perform fine-tuning training on the content generation model based at least on the first set of training samples and the second set of training samples to obtain a trained content generation model, wherein the trained content generation model is configured to generate content by following the plurality of rules based on a prompt input.

[0088] In some embodiments, each training sample in the first set of training samples includes a first sample prompt input and a corresponding first model response, each first sample prompt input including a question about a sub-rule included in a rule, and the corresponding first model response including a plurality of sub-rules included in the questioned rule.

[0089] In some embodiments, each training sample in the second set of training samples includes a second sample prompt input and a corresponding second model response, each second sample prompt input being configured to guide the content generation model to generate content based on a given sub-rule, and the corresponding second model response including sample content generated based on the given sub-rule.

[0090] In some embodiments, the apparatus 600 further includes a third sample generating module configured to generate, based on the plurality of rules and the plurality of sub-rules corresponding to each rule, a third set of training samples for the content generation model, the third set of training samples being configured to train the content generation model to determine a definition of a sub-rule included in a rule and to generate an example associated with the sub-rule; and the model training module 630 is further configured to perform the fine-tuning training on the content generation model based also on the third set of training samples.

[0091] In some embodiments, the third set of training samples includes a third sample prompt input and a corresponding third model response, each third sample prompt input being configured to guide the content generation model to generate a definition of a sub-rule included in a rule, and the corresponding third model response including the generated definition and a sample example.

[0092] In some embodiments, the apparatus 600 further includes a fourth sample generation module configured to generate, based on the plurality of rules and the plurality of sub-rules corresponding to each rule, a fourth set of training samples for the content generation model, the fourth set of training samples including fourth sample prompt inputs and corresponding fourth model responses, each fourth prompt input being configured to direct the content generation model to generate domain knowledge associated with the rule, and the corresponding fourth model response including the associated domain knowledge; and the model training module 630 is further configured to perform fine-tuning training on the content generation model further based on the fourth set of training samples.

[0093] In some embodiments, the plurality of sub-rules corresponding to each rule in the plurality of rules are independent of each other.

[0094] In some embodiments, the apparatus 600 further includes an indicator determination module configured to determine, based on the second set of training samples, at least one sample evaluation indicator for each of the plurality of rules; and an update module configured to update the second set of training samples and / or update at least one rule in the plurality of rules corresponding to a sub-rule based on the at least one sample evaluation indicator for each of the plurality of rules.

[0095] In some embodiments, the indicator determination module is further configured to, for a given rule in the plurality of rules, determine, from the second set of training samples, a total number of samples associated with the plurality of sub-rules corresponding to the given rule and a number of samples associated with each of the plurality of sub-rules, determine, based on a comparison between the number of samples associated with each of the plurality of sub-rules and the total number of samples, a first sample evaluation indicator for the plurality of sub-rules corresponding to the given rule, the first sample evaluation indicator indicating sample diversity of the plurality of sub-rules corresponding to the given rule.

[0096] In some embodiments, the indicator determination module is further configured to, for a given rule in the plurality of rules, determine, from the second set of training samples, a minimum number of samples and a maximum number of samples in the number of samples associated with each of the plurality of sub-rules corresponding to the given rule, determine, based on a comparison between the minimum number of samples and the maximum number of samples, a second sample evaluation indicator for the given rule, the second sample evaluation indicator indicating sample representativeness corresponding to the given rule.

[0097] The units and / or modules included in the apparatus 600 can be implemented utilizing various means including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units and / or modules can be implemented using software and / or firmware, e.g., machine-executable instructions stored on a storage medium. In addition to or alternatively, some or all of the units and / or modules in the apparatus 600 can be implemented at least partially by one or more hardware logic components. As an example and not by way of limitation, example types of hardware logic components that can be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SOCs), complex programmable logic devices (CPLDs), etc.

[0098] It should be understood that one or more steps in the above methods can be performed by an appropriate electronic device or combination of electronic devices. Such an electronic device or combination of electronic devices can include, for example, the electronic device 110 in FIG. 1.

[0099] FIG. 7 illustrates a block diagram of an electronic device 700 in which one or more embodiments of the present disclosure can be implemented. It should be understood that the electronic device 700 illustrated in FIG. 7 is merely an example and should not be construed to limit the functionality and scope of the embodiments described herein. The electronic device 700 illustrated in FIG. 7 can be used to implement the electronic device 110 of FIG. 1, or the apparatus 600 of FIG. 6.

[0100] As illustrated in FIG. 7, the electronic device 700 is in the form of a general electronic device. Components of the electronic device 700 can include, but are not limited to, one or more processors or processing units 710, a memory 720, a storage device 730, one or more communication units 740, one or more input devices 750, and one or more output devices 760. The processing unit 710 can be a real or virtual processor and capable of performing various processes according to programs stored in the memory 720. In a multi-processor system, multiple processing units perform computer-executable instructions in parallel to improve parallel processing capabilities of the electronic device 700.

[0101] The electronic device 700 typically includes a plurality of computer storage media. Such media can be volatile and / or nonvolatile, removable and / or non-removable, and can be implemented in any method or technology for storage of information and / or data. The memory 720 can be volatile (such as random access memory (RAM)), non-volatile (such as read-only memory (ROM), flash memory, or any other memory technology), or some combination of the multiple memory types. The storage device 730 can be a removable storage medium or a non-removable storage medium implemented in any method or technology for storage of information and / or data.

[0102] The electronic device 700 can further include additional removable / non-removable, volatile / non-volatile storage media. Although not shown in FIG. 7, a disk drive or other computer-readable media drive can be provided for reading from or writing to a removable, non-removable, volatile, or non-volatile computer-readable medium. In these instances, each drive can be connected to the bus (not shown) by one or more data media interfaces. The memory 720 can include a computer program product 725 having one or more program modules configured to carry out the various methods or actions of the various embodiments of the present disclosure.

[0103] The communication unit 740 enables communications with other electronic devices over a communication medium. Additionally, the functionality of the components of the electronic device 700 can be implemented in a single computing cluster or a plurality of computer machines that are capable of communicating with one another over a communication connection. As such, the electronic device 700 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network nodes.

[0104] The input device 750 can be one or more input devices, such as a mouse, a keyboard, a trackball, etc. The output device 760 can be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 700 can also communicate with one or more external devices (not shown) such as a storage device, a display device, etc. through the communication unit 740, as needed, one or more devices that enable a user to interact with the electronic device 700, or any device (e.g., a network card, a modem, etc.) that enables the electronic device 700 to communicate with one or more other electronic devices. Such communication can be carried out via an input / output (I / O) interface (not shown).

[0105] According to example implementations of the present disclosure, a computer-readable storage medium is provided having computer-executable instructions stored thereon, where the computer-executable instructions are executed by a processor to implement the method described above. According to example implementations of the present disclosure, a computer program product is also provided that is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, where the computer-executable instructions are executed by a processor to implement the method described above.

[0106] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0107] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0108] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0109] The computer program product of the second implementation can include a computer readable storage medium. The computer readable storage medium can include instructions. The instructions can include one or both of: instructions for causing a computer to enable a user equipment device to receive a configuration message from a base station, the configuration message comprising a configuration of a plurality of search space sets for a plurality of downlink control channel candidates; and instructions for causing a computer to enable a user equipment device to monitor the plurality of downlink control channel candidates in accordance with the configuration of the plurality of search space sets.

[0110] Various implementations of the disclosure have been described in detail above. The foregoing description is exemplary and is not intended to be exhaustive or to be limited to the precise implementations disclosed. Numerous modifications and variations are possible in light of the above teachings without departing from the scope and spirit of the disclosure. The above description is meant to be exemplary only and is not intended to be limiting. The terms used in this document are to be interpreted in their broadest reasonable manner, such that the principles and applications described herein can be used in many different implementations.

Claims

1. A method for rule alignment of a model, comprising: obtaining, for each rule in a plurality of rules, a plurality of sub-rules corresponding to each rule, each sub-rule being used to describe one aspect in a plurality of aspects of the corresponding rule; generating, based on the plurality of rules and the plurality of sub-rules corresponding to each rule, a first set of training samples and a second set of training samples for a pre-trained content generation model, wherein the first set of training samples is configured to train the content generation model to determine a correspondence between a sub-rule and a rule, and the second set of training samples is configured to train the content generation model to generate content based on a sub-rule; and performing fine-tuning training on the content generation model based at least on the first set of training samples and the second set of training samples to obtain a trained content generation model, wherein the trained content generation model is configured to generate content by following the plurality of rules based on a prompt input.

2. The method of claim 1, wherein each training sample in the first set of training samples comprises a first sample prompt input and a corresponding first model response, each first sample prompt input comprising a question on a sub-rule contained in a rule, and the corresponding first model response comprising a plurality of sub-rules contained in the questioned rule.

3. The method of claim 1, wherein each training sample in the second set of training samples comprises a second sample prompt input and a corresponding second model response, each second sample prompt input being configured to direct the content generation model to generate content based on a given sub-rule, and the corresponding second model response comprising sample content generated based on the given sub-rule.

4. The method of claim 1, further comprising: generating, based on the plurality of rules and the plurality of sub-rules corresponding to each rule, a third set of training samples for the content generation model, the third set of training samples being configured to train the content generation model to determine a definition of a sub-rule contained in a rule and to generate an example associated with the sub-rule; and wherein performing fine-tuning training on the content generation model comprises performing fine-tuning training on the content generation model further based on the third set of training samples.

5. The method of claim 4, wherein the third set of training samples comprises a third sample prompt input and a corresponding third model response, each third sample prompt input being configured to direct the content generation model to generate a definition of a sub-rule contained in a rule, and the corresponding third model response comprising the generated definition and a sample example.

6. The method of claim 1, further comprising: generating, based on the plurality of rules and the plurality of sub-rules corresponding to each rule, a fourth set of training samples for the content generation model, the fourth set of training samples comprising a fourth sample prompt input and a corresponding fourth model response, each fourth prompt input being configured to direct the content generation model to generate domain knowledge associated with a rule, and the corresponding fourth model response comprising the associated domain knowledge; And wherein performing fine-tuning training on the content generation model comprises performing fine-tuning training on the content generation model further based on the fourth set of training samples. 7.The method of claim 1, wherein the plurality of sub-rules corresponding to each of the plurality of rules are independent of each other. 8.The method of claim 1, further comprising: determining at least one sample evaluation indicator for each of the plurality of rules based on the second set of training samples; and updating the second set of training samples and / or updating at least one of the plurality of rules corresponding to a plurality of sub-rules based on the at least one sample evaluation indicator for each of the plurality of rules.

9. The method of claim 8, wherein determining, based on the second set of training samples, at least one sample evaluation metric for each of the plurality of rules comprises: For a given rule of the plurality of rules, determining a total number of samples associated with the plurality of sub-rules corresponding to the given rule and a number of samples associated with each of the plurality of sub-rules from the second set of training samples, determining a first sample evaluation indicator of the plurality of sub-rules corresponding to the given rule based on a comparison between the number of samples associated with each of the plurality of sub-rules and the total number of samples, the first sample evaluation indicator indicating a sample diversity of the plurality of sub-rules corresponding to the given rule.

10. The method of claim 8, wherein determining, based on the second set of training samples, at least one sample evaluation metric for each of the plurality of rules comprises: For a given rule of the plurality of rules, determining a minimum number of samples and a maximum number of samples from the number of samples associated with each of the plurality of sub-rules corresponding to the given rule from the second set of training samples, determining a second sample evaluation indicator of the given rule based on a comparison between the minimum number of samples and the maximum number of samples, the second sample evaluation indicator indicating a sample representativeness of the given rule. 11.An apparatus for rule alignment of a model, comprising: a sub-rule obtaining module configured to, for each of a plurality of rules, obtain a plurality of sub-rules corresponding to each of the rules, each of the sub-rules being used to describe one of a plurality of aspects of the corresponding rule; a sample generating module configured to generate, based on the plurality of rules and the plurality of sub-rules corresponding to each of the rules, a first set of training samples and a second set of training samples for a pre-trained content generation model, wherein the first set of training samples are configured to train the content generation model to determine a correspondence between a sub-rule and a rule, and the second set of training samples are configured to train the content generation model to generate content based on a sub-rule; and a model training module configured to perform fine-tuning training on the content generation model based on at least the first set of training samples and the second set of training samples to obtain a trained content generation model, wherein the trained content generation model is configured to generate content by following the plurality of rules based on a prompt input. 12.An electronic device, comprising: at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the electronic device to perform the method according to any one of claims 1 to 10.

13. A computer-readable storage medium having stored thereon a computer program, the computer program being executable by a processor to implement the method of any one of claims 1 to 10.

14. A computer program product tangibly stored in a computer storage medium and comprising computer-executable instructions that, when executed by a device, cause the device to perform the method of any one of claims 1 to 10.

Images