Automated assessment of consistency with human preferences

Abstract

There is described a computing device (500) for evaluating a scoring model (302), the computing device (500) for: for each of a plurality of items (204a, 204b, 204c, 204d) in a set of items (204), obtaining an item score (205a, 205b, 205c, 205d); for each item of the plurality of items (204a, 204b, 204c, 204d) in the item set (204), obtaining a score (303a, 303b, 303c, 303d) generated by a scoring model, the score (303a, 303b, 303c, 303d) generated by the scoring model being determined by a scoring model (302); associating the item score (205a, 205b, 205c, 205d) with a score (303a, 303b, 303c, 303d) generated by the scoring model; and evaluating the scoring model (302) according to the correlation (301). In this manner, the accuracy of the scoring model (302) may be evaluated based on a comparison with the item scores (205a, 205b, 205c, 205d).

Description

Technical Field

[0001] This invention relates to an apparatus and method for evaluating a scoring model. Background Technology

[0002] In both business and research, it is generally believed that the best Language Model / Personal Assistant (LM / PA) should align with Human Preference (HP). Currently, efficiently and automatically evaluating HP remains an unsolved problem. Figure 1 This is a table comparing the main differences between prior art methods and this application. The table summarizes and compares the most relevant methods in LM / PA evaluation compared to this system, breaking them down into multiple steps (from left to right). Detailed information about HumanRankEval (this system) is described in this document.

[0003] All methods in the table may generate a numerical score to evaluate a specific aspect (objective) of the LM / PA. The "Reviewer" column specifies who determines the correct answer (benchmark truth). This could be a human or human-generated content (as an agent), such as answers to multiple-choice exams, programming tasks, tool / API usage, and various other ways of testing LM / PA competence. In other cases, the LM / PA itself acts as the reviewer. The "Indicators" column specifies how the final score is calculated based on the reviewer's output, for example, calculating the accuracy of a multiple-choice exam or the pass rate@1 for generating correct code for a given description in the LM / PA.

[0004] Existing technology systems can be divided into the following three categories: Human evaluation – This involves presenting the LM / PA output to users and asking them to rate one or more aspects of the HP, such as usefulness, offensiveness, factual accuracy, clarity, etc.; or (b) asking humans to choose their preferred output (typically from two different LM / PAs); or (c) any combination of (a) and (b), where humans directly evaluate the quality of the LM / PA [Judging LLM-as-a-judge with MT-Bench and Chatbot Arena; and BeaverTails: Towards Improved Safety Alignment of LLM via a Human-PreferenceDataset]. HP is typically converted into an Elo score or win percentage (the score is relative to other LM / PAs).

[0005] Automated evaluation (compared to human text) – These methods may involve evaluation tasks that assess well-defined correct solutions to one or more human-generated texts. Tasks typically test massive multitask language understanding, functional (code) correctness (evaluating Large Language Models Trained on Code), success rate (AgentBench: Evaluating LLMs as Agents), and similar accuracy metrics across a range of tasks, such as tool / API usage (AgentBench: Evaluating LLMs as Agents), complex logical reasoning (Training verifiers to solve mathword problems), and school entrance exams and math competitions (Agieval: A human-centric benchmark for evaluating foundation models). The goal is not to test HP, however, there are some correlations between HP and these metrics.

[0006] Automated evaluation (comparing LM / PA text) – An emerging trend in automated evaluation is the use of (often very large and proprietary) LM / PAs, such as ChatGPT or GPT-4, as a faster alternative to human judgment. LM / PAs are prompted to rate the outputs of other LM / PAs [Judging LLM-as-a-judge with MT-Bench and Chatbot Arena; and AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback], with ratings including (a) aspects of human preference, such as usefulness, utility, safety, etc., or (b) choosing one from multiple responses. HP is converted to an Elo score or win percentage (again, the score is relative to other LM / PAs).

[0007] Existing technology systems may have the following drawbacks: Human evaluation – This process can be costly in terms of time and money (organization and coordination). Finding suitable experts across multiple domains can be challenging, and human reviewers may be inconsistent and / or biased [Style OverSubstance: Evaluation Biases for Large Language Models]. This approach may not allow for rapid model iteration. Human evaluation requires testing multiple models because Elo scores (or win rates) are relative to other LMs / PAs (there are no absolute scores).

[0008] Automatic evaluation (compared to human text) – Different tasks assess specific capabilities of LM / PA, which helps test these behaviors in the model; however, they may not accurately estimate HP. Higher scores on these metrics do not necessarily mean that humans prefer these LM / PAs.

[0009] Automated evaluation (comparing with LM / PA text) – This method (almost always) uses very large and proprietary LM / PAs, such as GPT-4 or ChatGPT. These LM / PAs may be uncontrollable; they may (and have been) shut down or changed in the short term; there may be transparency issues, as we may not know any details of the training. Data privacy may be a concern. Consistency with human performance has not been proven.

[0010] In view of this, there is a need to develop a device and method to overcome the above problems. Summary of the Invention

[0011] According to a first aspect, a computing device is provided for evaluating a scoring model, the computing device including one or more processors and a memory storing data (the data defining program code executable by the one or more processors) in a non-transient form, wherein the program code is executable by the one or more processors, such that the computing device is configured to: obtain a project score for each project in a plurality of projects in a set of projects; obtain a score generated by a scoring model for each project in the plurality of projects in the set of projects, the score generated by the scoring model being determined by the scoring model; correlate the project score with the score generated by the scoring model; and evaluate the scoring model based on the correlation. In this way, the accuracy of the scoring model can be evaluated based on a comparison with the project score.

[0012] In some implementations, each of the plurality of projects may include the output of the project model. In some implementations, the project set may include multiple projects generated based on the same input to the project model. In this way, the accuracy of the scoring model can be evaluated when scoring the project model.

[0013] In some implementations, the project score may include one or more constituent elements. In some implementations, each constituent element may include a characteristic of the project. In some implementations, the project score may include a composite score of the constituent elements. In some implementations, the constituent elements may be based on one or more of the following aspects of the project: usefulness, usability, security, timeliness, clarity, and factual accuracy. In this way, the score can be broken down to cover different aspects of the project. The composition of the constituent elements can provide a simple summary of various aspects of the project.

[0014] In some implementations, the item rating may be based on human preferences. In some implementations, the item rating may be determined by a human. In some implementations, the item rating may be determined by an item rating model. In this way, the rating model can be evaluated based on its ability to rate human preferences, and optionally based on ratings determined by humans. Alternatively, the rating model can be evaluated based on ratings determined by the item rating model.

[0015] In some implementations, the ratings generated by the rating model may include simulated probabilities of the items as determined by the rating model, reflecting human preferences. In this way, the rating model can be evaluated based on its ability to simulate human choices.

[0016] In some implementations, the computing device can be used to: select a subset of items from the item set based on the item ratings, and obtain the ratings generated by the rating model only for the selected subset of items in the item set. In this way, the items can be filtered, so that the rating model only needs to determine ratings for the subset of items, thereby improving efficiency.

[0017] In some implementations, the computing device can be used to associate the project score with the score generated by the rating model based on the product of the project score and the score generated by the rating model. In some implementations, the computing device can be used to associate the project score with the score generated by the rating model based on a correlation function between the project score and the score generated by the rating model. In some implementations, the correlation function can be a Pearson correlation. In some implementations, the correlation function can be a Spearman correlation. This allows for efficient association.

[0018] In some implementations, the computing device can be used to generate the plurality of projects using the project model. In this way, the device can generate the projects independently, thus constituting a self-sufficient evaluation device.

[0019] In some implementations, the computing device can be used to determine the scores generated by the scoring model based on the plurality of items. In this way, the device can generate the scores generated by the scoring model itself, thus constituting a self-sufficient evaluation device.

[0020] In some implementations, the project score may not be determined by the scoring model. In this way, the scoring model's evaluation will not be biased towards giving itself a good assessment.

[0021] In some implementations, the computing device can be used to obtain multiple item ratings for each of the plurality of items in the item set. In some implementations, each of the multiple item ratings for the items in the item set can be determined by different people. In this way, the evaluation can be based on a range of different human preferences, which can test whether the rating model can take into account different human types.

[0022] In some implementations, the computing device can be used to perform the steps described in any of the foregoing statements for multiple inputs to the project model, in order to evaluate the scoring model based on all of the multiple inputs. In this way, the evaluation can be based on a range of different inputs to the project model, which can test whether the scoring model can take into account different input types.

[0023] In some implementations, the item model may include a language model, the output item of the item model may include text, and the score generated by the scoring model may be a numerical score or an ordinal score. In some implementations, the scoring model may include a language model. In some implementations, the language model may include one or more of the following: neural models, artificial models, statistical models, and multimodal language models. In this way, the scoring model evaluation can be applied to a language model setting.

[0024] In some implementations, the project model may include one or more of the following: an image generation model, a sound generation model, a video generation model, and a physical project generation model. In some implementations, the scoring model may include one or more of the following: an image model, a sound model, a video model, and a physical project model. In this way, the scoring model evaluation can be applied to non-verbal model settings.

[0025] In some implementations, the computing device can be used to output the evaluation of the scoring model. In this way, the evaluation can be used for more purposes. In some implementations, the computing device can be used to train the scoring model using the evaluation of the scoring model. In this way, the evaluation can be used to improve the scoring model. In some implementations, the computing device can be used to train different models using the evaluation of the scoring model. In this way, the evaluation can be used to improve different models.

[0026] According to a second aspect, a method for evaluating a scoring model is provided, the method comprising the steps of: obtaining a project score for each project among a plurality of projects in a project set; obtaining a score generated by a scoring model for each project among the plurality of projects in the project set, the score generated by the scoring model being determined by the scoring model; associating the project score with the score generated by the scoring model; and evaluating the scoring model based on the correlation. In this manner, the accuracy of the scoring model can be evaluated based on a comparison with the project score. Attached Figure Description

[0027] The invention will now be described by way of example with reference to the accompanying drawings. In the drawings: Figure 1 A table comparing the differences between prior art methods and this application is shown.

[0028] Figure 2 The project generation pipeline of this application is illustrated schematically.

[0029] Figure 3 The scoring model evaluation pipeline of this application is illustrated schematically.

[0030] Figure 4 An example of a computer implementation method for evaluating a scoring model is shown.

[0031] Figure 5 An example of an apparatus for performing the methods described herein is shown. Detailed Implementation

[0032] The apparatus and methods described in this paper relate to the evaluation of scoring models.

[0033] Embodiments of the present invention can solve one or more of the problems mentioned above by: for each item in a set of multiple items, obtaining the score generated by the scoring model, and evaluating the scoring model based on the correlation between the item score and the score generated by the scoring model. In this way, the accuracy of the scoring model can be evaluated based on a comparison with the item score.

[0034] The following terms may be used throughout the application: Language Model (LM): A mechanism that represents language (natural and / or artificial) as a probability distribution over words / tokens, capable of outputting a sequence of texts and possibly the probability of individual words / tokens in those text sequences.

[0035] Personal Assistant (PA): Typically (but not necessarily) tuned / trained to follow instructions in order to perform a variety of tasks (usually to assist humans), such as coding, writing, answering questions, summarizing, and other related tasks (e.g., ChatGPT or GPT-4).

[0036] Human Preference (HP): This refers to the quality / type of output received from PA that humans prefer, usually expressed as choosing one from multiple outputs / answers. Note: Humans generally prefer outputs that are useful, practical, non-offensive, clear, safe, correct, and factual. However, current systems may treat HP as a black box, without attribution, and may even require knowledge of the aforementioned component ratings to evaluate HP.

[0037] The steps in this application can be... Figure 1 The HumanRankEval line in the table illustrates this. Compared to previous studies, the various steps of HumanRankEval may provide a better assessment of HP.

[0038] This system can solve the following problems: How can HP be represented / captured automatically? Manual collection (e.g., through crowdsourcing) can be extremely challenging because it requires capturing the naturalness, spontaneity, and openness of output carefully rated by thousands of enthusiasts and experts on real-world topics / issues.

[0039] How can we assign ratings to LM / PA that are highly correlated with human preferences? It would be desirable to have an efficient method to determine this and adjust model training accordingly.

[0040] This system offers the following advantages: It can capture / represent HP media.

[0041] This can eliminate the cost of crowdsourcing such data.

[0042] This can eliminate the risk of collecting non-natural data (it is difficult to find enough domain experts to mimic such open-ended questions and answers).

[0043] The process of accurately estimating the consistency between LM / PA and HP.

[0044] HumanRankEval scores are likely to align well with human preferences (unlike common automated benchmarks or LLMs-as-judge).

[0045] It can provide complete control, transparency, and privacy (unlike using proprietary and constantly changing APIs).

[0046] Sampling from LM / PA may not be necessary (it is usually random), making replication fast and stable.

[0047] This system can solve the above problems through the following steps: Compile datasets (from public or private sources) in a novel format capable of capturing / representing HP. This might manifest as multiple answers to information-seeking questions, each rated by (on average) dozens of experts in the field. Users can interact naturally with each other and collaboratively determine the ratings for each answer, which can then be used to calculate our new metrics.

[0048] The probability of each answer (text, completion, or output) is collected under its current distribution (training weights) to query the scoring model LM / PA.

[0049] Human ratings are correlated with model probabilities to obtain the HumanRankEval score. A higher score indicates that the rating model is more consistent with human preferences (expected outcomes).

[0050] HumanRankEval aims to quickly and accurately evaluate the training / development progress (LM / PA) automatically at any stage of its development to estimate the degree of consistency between the LM / PA and the HP (the optimal PA has high consistency). This metric can be used to tune training / development in an efficient and cost-effective manner.

[0051] HumanRankEval can access LM / PA output, i.e., token / word probabilities, rather than just providing text content as in web-based APIs. Therefore, the HumanRankEval metric may be applicable to LM / PA owners and / or open-source LM / PAs where such probabilities can be easily queried. Furthermore, HP captures the multidimensional characteristics of LM / PAs, while HumanRankEval may only provide a score for that overall score, rather than for its constituent elements such as usefulness, usability, security, timeliness, clarity, etc.

[0052] HumanRankEval can be used in products that utilize LM / PA, such as programming or writing assistants (Copilot). Its greatest value lies in saving time and cost compared to the slow and expensive process of developing and evaluating LM / PA by obtaining human feedback. HumanRankEval is not a replacement for human judgment (e.g., the final decision on whether to deploy LM / PA to end users), but its purpose is to accelerate the iterative development of LM / PA through frequent evaluations (and experimentation) that are highly consistent with known HP.

[0053] The HumanRankEval system may include the following steps: (1) obtaining human ratings (data preparation), (2) obtaining LM / PA probabilities, and (3) correlating them to calculate the HumanRankEval score. The goal may be to assess whether LM / PA is consistent with HP.

[0054] Figure 2 The project generation pipeline 200 of this application is illustrated schematically.

[0055] Multiple items 204a, 204b, 204c, and 204d can be generated. These multiple items 204a, 204b, 204c, and 204d can form an item set 204. Each item in item set 204 (204a, 204b, 204c, and 204d) shares a common feature. Each item in item set 204a, 204b, 204c, and 204d is generated from the same input 203. Figure 2 The example shows four items: 204a, 204b, 204c, and 204d. However, the number of items 204a, 204b, 204c, and 204d can vary depending on the implementation.

[0056] Each of the multiple projects 204a, 204b, 204c, and 204d may include an output from project model 201. Input 203 can be input into project model 201, and multiple projects 204a, 204b, 204c, and 204d can be output. Each of the multiple projects 204a, 204b, 204c, and 204d can be generated from the same input 201. Project model 201 can form part of an apparatus that generates multiple projects 204a, 204b, 204c, and 204d. Alternatively, project model 201 can be separate from the apparatus, and the multiple projects 204a, 204b, 204c, and 204d can be received by the apparatus.

[0057] exist Figure 2In the example shown, input 203 may include question 203. Output items may include answers 204a, 204b, 204c, and 204d. Question 203 may have different answers 204a, 204b, 204c, and 204d. In this case, item model 201 may include a language model. Therefore, input 203 and output 204 include language. More generally, when item model 201 includes a language model, the output item 204 of item model 201 may include text. The language model may include one or more of the following: neural models, artificial models, statistical models, and multimodal language models.

[0058] Alternatively, project model 201 may include a non-linguistic model. Project model 201 may include one or more of the following: an image generation model, a sound generation model, a video generation model, and a physical project generation model. In this case, output 204 may include an image, sound, video, or physical project, respectively. The physical project may be in the form of a 3D printed output.

[0059] Without using project model 201, projects 204a, 204b, 204c, and 204d may have been generated in other ways. For example, projects 204a, 204b, 204c, and 204d could be physical objects or products. Subsequently, projects 204a, 204b, 204c, and 204d can be scored as described herein.

[0060] Multiple item ratings 205a, 205b, 205c, and 205d can be obtained. Item ratings 205a, 205b, 205c, and 205d can correspond to multiple items 204a, 204b, 204c, and 204d in item set 204. For each item in item set 204, there can be an item rating 205.

[0061] Multiple item ratings 205a, 205b, 205c, and 205d can be obtained without generating multiple item ratings 205a, 205b, 205c, and 205d. In other words, multiple item ratings 205a, 205b, 205c, and 205d can be received by the device from another system. Multiple item ratings 205a, 205b, 205c, and 205d can be generated by humans. For each of the multiple items 204a, 204b, 204c, and 204d in item set 204, a human can generate an item rating 205. In other words, a human may have already assigned an item rating 205 to each of the multiple items 204a, 204b, 204c, and 204d.

[0062] Alternatively, multiple item ratings 205a, 205b, 205c, and 205d can be generated by an item rating model. For each of the multiple items 204a, 204b, 204c, and 204d in the item set 204, the item rating model can generate an item rating 205. In other words, the item rating model may have already assigned an item rating 205 to each of the multiple items 204a, 204b, 204c, and 204d. The item rating model can include any rating machine. The item rating model can include a machine learning model. The item rating model can output item ratings 205a, 205b, 205c, and 205d to the device. Alternatively, item model 201 can be part of the device, and the item model as part of the device can generate multiple item ratings 205a, 205b, 205c, and 205d.

[0063] For each of the multiple items 204a, 204b, 204c, and 204d in item set 204, multiple item ratings 205a, 205b, 205c, and 205d can be obtained. Given that a human determines item ratings 205a, 205b, 205c, and 205d, multiple different humans can independently determine item rating 205 for each of the multiple items 204a, 204b, 204c, and 204d in item set 204. Furthermore, given that the item rating model determines item ratings 205a, 205b, 205c, and 205d, multiple different item rating models can independently determine item rating 205 for each of the multiple items 204a, 204b, 204c, and 204d in item set 204. Alternatively, for each of the multiple items 204a, 204b, 204c, 204d in item set 204, the same item scoring model can determine multiple item scores 205. In subsequent cases, for each item score 205 of the different item scores 205 for the same items 204a, 204b, 204c, 204d, the item scoring model can use different criteria or indicators.

[0064] Item rating 205 may include one or more constituent elements. Each constituent element may include a characteristic of item 205. In other words, item rating 205 may be based on one or more values ​​associated with the characteristics of item 205. Item rating 205 may include a composite score of the constituent elements. Item rating 205 may be composed of each constituent element. Item rating 205 may include the sum of the constituent elements. Item rating 205 may include the product of the constituent elements. In this way, the composition of the constituent elements can provide a simple summary of various aspects of the item. The constituent elements may be based on one or more of the following aspects of item 205: usefulness, utility, safety, timeliness, clarity, and factual accuracy. For example, item rating 205 may include 6 points (usefulness), 2 points (utility), 5 points (safety), 3 points (timeliness), 5 points (clarity), and 4 points (factual accuracy), for a total score of 25, for example, item rating 205a. In this way, item rating 205 can be customized according to human preferences.

[0065] A subset of items 204 can be selected. This subset of items 204 can be selected by filter 202. The subset of items 204 can be filtered based on their item ratings 205. For example, items 204 with the lowest item rating 205 can be filtered out. Items 204 with item ratings 205 below a certain threshold can be filtered out. Alternatively, a certain percentage of items among the items 204 with lower item ratings 205 can be filtered out. Figure 2 In the example shown, item 204, which has four highest item scores of 205, is retained after filtering. This way, item 204 can be retained more accurately based on its item score of 205.

[0066] Figure 2 The details illustrate a specific embodiment of the project generation pipeline 200. Human-generated data (7000 questions in total) can be collected from public communities (StackOverflow, StackExchange). The data can be filtered 202 to select high-quality pairs (each answer must have at least 5 votes from domain experts, each question must have at least 40 votes, each question-answer pair must have a maximum of 4000 characters, no duplicates, etc.). The device can retain the top 4 answers 204 and their ratings 205.

[0067] The source of the content, the number of questions and answers (or sentences, or any other multimodal content), and the details of filtering may vary. The quality score for the answer / output can be represented numerically, such as an integer or a floating-point number.

[0068] Figure 3 The scoring model evaluation pipeline of this application is illustrated schematically.

[0069] Multiple rating models can be used to generate ratings 303a, 303b, 303c, and 303d. These ratings 303a, 303b, 303c, and 303d can correspond to multiple items 204a, 204b, 204c, and 204d in item set 204. For each item in item set 204, a rating model-generated rating 303 can exist.

[0070] The scores 303a, 303b, 303c, and 303d generated by multiple scoring models can be obtained without generating scores 303a, 303b, 303c, and 303d from multiple scoring models. In other words, the scores 303a, 303b, 303c, and 303d generated by multiple scoring models can be received by the device from another system. The scores 303a, 303b, 303c, and 303d generated by multiple scoring models can be generated by scoring model 302. For each of the multiple items 204a, 204b, 204c, and 204d in the item set 204, scoring model 302 can generate scores 303a, 303b, 303c, and 303d generated by the scoring model. In other words, scoring model 302 may have already assigned a score 303 generated by the scoring model to each of the multiple items 204a, 204b, 204c, and 204d. Scoring model 302 may include a machine learning model. The scoring model 302 can output scores 303a, 303b, 303c, and 303d generated by the scoring model to the device. Alternatively, the scoring model 302 can be part of the device, and the scoring model 302 as part of the device can generate multiple scores 303a, 303b, 303c, and 303d generated by the scoring model.

[0071] The ratings 303a, 303b, 303c, and 303d generated by the rating model can include numerical ratings or ordinal ratings. In the case of numerical ratings, the ratings 303a, 303b, 303c, and 303d generated by the rating model can include simulated probabilities of human preference items 204a, 204b, 204c, and 204d determined by rating model 302. In other words, the ratings 303a, 303b, 303c, and 303d generated by the rating model can give the probability that a human will choose item 204a, 204b, 204c, or 204d among all items 204a, 204b, 204c, and 204d. The probability value can be based on 1. The sum of the total probabilities of the ratings 303a, 303b, 303c, and 303d generated by all rating models can be 1. Therefore, the probability of each rating 303a, 303b, 303c, and 303d generated by each rating model is equal to or less than 1. like Figure 2As shown in the example, the ratings generated by the rating model for 303a, 303b, 303c, and 303d are 0.5, 0.11, 0.1, and 0.02, respectively. Item 204a has the highest rating of 0.5 generated by the rating model, making it the most likely to be chosen by humans, as determined by rating model 302. Alternatively, a log-likelihood probability value can be used instead of a probability value based on 1. In the case of ordinal ratings, rating model 302 can rank each of items 204a, 204b, 204c, and 204d from best to least preferred based on the human preference determined by rating model 302.

[0072] exist Figure 2 and Figure 3 In the example shown, input 203 may include question 203. Output items may include answers 204a, 204b, 204c, and 204d. Question 203 may have different answers 204a, 204b, 204c, and 204d. In this case, scoring model 302 may include a language model. Therefore, scoring model 302 may be able to score items 204a, 204b, 204c, and 204d that include text. The language model may include one or more of the following: neural models, artificial models, statistical models, and multimodal language models.

[0073] Alternatively, if item model 201 is not a non-verbal model, scoring model 302 can be a non-verbal model. Scoring model 302 can include one or more of the following: image model, sound model, video model, and physical item model. Therefore, scoring model 302 can be able to score items 204a, 204b, 204c, and 204d, which can respectively include image, sound, video, or physical items. Physical items can be in the form of 3D printed outputs.

[0074] Project score 205 may not be determined by scoring model 302. In other words, scoring model 302 may not be the same model as the project scoring model. Therefore, scoring model 302 is not used to generate project score 205 and the score 303 generated by the scoring model. In this way, the evaluation of scoring model 302 will not be biased towards giving itself a good evaluation.

[0075] Item ratings 205a, 205b, 205c, and 205d, and ratings 303a, 303b, 303c, and 303d generated by the rating model, can be correlated. In this way, ratings 303a, 303b, 303c, and 303d generated by rating model 302 can be compared with item ratings 205a, 205b, 205c, and 205d. In the case of human-generated item ratings 205a, 205b, 205c, and 205d, ratings 303a, 303b, 303c, and 303d generated by rating model 302 can be compared with human preferences.

[0076] like Figure 3 As shown, item scores 205a, 205b, 205c, and 205d, and scores 303a, 303b, 303c, and 303d generated by the scoring model, can be correlated based on the correlation function 301 of item scores 205a, 205b, 205c, and 205d and scores 303a, 303b, 303c, and 303d generated by the scoring model. In other words, for each item in items 204a, 204b, 204c, and 204d, the correlation function 301 of item scores 205a, 205b, 205c, and 205d and scores 303a, 303b, 303c, and 303d generated by the scoring model can be used. The correlation function 301 can include Pearson correlation. The correlation function 301 can include Spearman correlation.

[0077] Alternatively, item scores 205a, 205b, 205c, 205d and scores 303a, 303b, 303c, 303d generated by the scoring model can be correlated based on the product of item scores 205a, 205b, 205c, 205d and scores 303a, 303b, 303c, 303d generated by the scoring model. In other words, for each of items 204a, 204b, 204c, 204d, the correlation is the product of item scores 205a, 205b, 205c, 205d and scores 303a, 303b, 303c, 303d generated by the scoring model.

[0078] The scoring model 302 can be evaluated by assessing the correlation between project scores 205a, 205b, 205c, and 205d and the scores 303a, 303b, 303c, and 303d generated by the scoring model. The closer the scores 303a, 303b, 303c, and 303d generated by the scoring model are to the optimal project scores 205a, 205b, 205c, and 205d, the better the evaluation is likely to be. The evaluation may include assessment scores. Assessment scores can provide indicators of the accuracy or applicability of the scoring model 302.

[0079] refer to Figure 2 and Figure 3 The process described in this paper can be used for multiple inputs 203 of the project model. In other words, multiple different inputs 203 can be used. Each input 203 can generate multiple outputs 204. A project score 205 and a score 303 generated by the scoring model can be obtained and / or generated for each output 204 generated from each input 203. Each input 203 can be used simultaneously or sequentially. The scoring model 302 can be evaluated based on multiple inputs 203. Figure 2 and Figure 3 The example of question 203 and answer 204 shows that multiple questions 203 can be posed to the project model 201, thereby generating multiple answers 204 for each question 203. In this way, a wider range of inputs 203 can be used to evaluate the scoring model 203.

[0080] The evaluation of rating model 302 can be used for a range of other purposes. The evaluation of rating model 302 can be output. The evaluation of rating model 302 can be output for use by different systems. For example, multiple different rating models 302 can be evaluated and compared. In this way, the rating model 302 to be used can be optimized.

[0081] The evaluation of rating model 302 can be used to train rating model 302. In the case of machine learning rating model 302, the evaluation of rating model 302 can be used to train rating model 302 through backpropagation. Training of rating model 302 based on the evaluation of rating model 302 can be automatic or manual. During the use of rating model 302, the training of rating model 302 can be continuous or periodic. The evaluation of rating model 302 can also be used to train different models. For example, other rating models can be trained based on the evaluated rating model 302.

[0082] Figure 3 The details illustrate a specific embodiment of the scoring model evaluation pipeline 300. This system can obtain a probabilistic score 303 for each answer 204 (or text sequence / output) from the LM / PA 302. Raw scores (logits) can be collected from the LM / PA 302 for each word in the answer 204. Logarithmic and softmax functions can be used, and then the average of these probabilities or log-likelihoods 303 (using characters instead of the word sequence length) can be used.

[0083] Human scores (25, 17, 11, 5)205 can be correlated with LM / PA probabilities (0.5, 0.11, 0.1, 0.02)303 to obtain a HumanRankEval score. The correlation 301 can be calculated using equivalent methods to the Pearson coefficient, such as the Spearman Rank correlation coefficient or other similar correlation functions (including using metrics of similar accuracy where applicable). A higher HumanRankEval score may indicate greater agreement with HP.

[0084] This system can be adjusted in the following ways: The LM / PA probability 303 for each answer 204 (or any text sequence, or any output) can be obtained without accessing individual word / symbol scores. For example, all answers 204 can be provided in the prompt, and the LM / PA 302 can be requested to provide scores for all answers in its output (this method works for API-based LM / PA, such as ChatGPT).

[0085] HumanRankEval can be extended to include HP components such as usefulness, security, clarity, factual accuracy, practicality, timeliness, etc. (if available).

[0086] HumanRankEval can be extended to evaluate specialized topics and / or multilingual applications (natural language and / or programming languages). This is not necessarily limited to matching human preferences for these topics, but can be applied to matching any preference criteria.

[0087] HumanRankEval can be extended beyond language output to evaluate any type of preference-based output, such as images or sound (multimodal). Examples include evaluating stable diffusion images of HP, text-to-speech (TTS) output, or some other generative AI content (music, video).

[0088] HumanRankEval can be used as a training loss, i.e., by backpropagating the correlation between human rating 205 and LM / PA probability 303 during instruction tuning.

[0089] HumanRankEval can use deterministic generation to collect LM / PA (instead of human) answers / outputs with known quality 204, and then associate their probabilities, i.e., score the LM / PA outputs.

[0090] Figure 4Examples of methods 400 for evaluating a scoring model are summarized below. In step 401, method 400 includes: for each item in a plurality of items in a project set, obtaining an item score. In step 402, method 400 includes: for each item in the plurality of items in the project set, obtaining a score generated by a scoring model, the score generated by the scoring model being determined by the scoring model. In step 403, method 400 includes: associating the item scores with the scores generated by the scoring model. In step 404, method 400 includes: evaluating the scoring model based on the correlation.

[0091] Figure 5 An example of a device 500 for implementing method 400 is schematically shown. The computing device 500 may include pipelines 200 and 300. The device 500 may be implemented on an electronic device, such as a computer, laptop, tablet, or smartphone.

[0092] Apparatus 500 includes a processor 501 for processing a dataset in the manner described herein. For example, processor 501 may be implemented as a computer program running on a programmable device such as a central processing unit (CPU). Apparatus 500 includes a memory 502 for communicating with processor 501. Memory 502 may be non-volatile memory. Processor 501 may also include a cache (…). Figure 5 (Not shown in the image) for temporarily storing data from memory 502. Apparatus 500 may include more than one processor 501 and more than one memory 502. Memory 502 may store data executable by processor 501. Processor 501 may be used to operate according to a computer program stored in a machine-readable storage medium in a non-transitory form. The computer program may store instructions that cause the processor to perform its methods in the manner described herein. The method steps described herein may be performed by a computer-readable storage medium. The method steps described herein may be performed by a computer program product.

[0093] The applicant hereby discloses each individual feature described herein, as well as any combination of two or more such features, which, to the ordinary knowledge of those skilled in the art, can be implemented as a whole according to this specification, regardless of whether such features or combinations of features solve any problem disclosed herein, and without limiting the scope of the claims. The applicant notes that various aspects of the invention may include any such individual feature or combination of features. In view of the foregoing description, various modifications that can be made within the scope of the invention will be apparent to those skilled in the art.

Claims

1. A computing device (500) for evaluating a scoring model (302), characterized in that, The computing device (500) includes one or more processors (501) and a memory (502) that stores data in a non-transient form, the data defining program code executable by the one or more processors (501), wherein the program code is executable by the one or more processors (501) such that the computing device (500) is used to: for each of a plurality of items (204a, 204b, 204c, 204d) in a set of items (204), obtain an item score (205a, 205b, 205c, 205d). For each of the multiple items (204a, 204b, 204c, 204d) in the item set (204), obtain the scores (303a, 303b, 303c, 303d) generated by the scoring model, which is determined by the scoring model (302); The project scores (205a, 205b, 205c, 205d) are associated with the scores (303a, 303b, 303c, 303d) generated by the scoring model. The scoring model (302) is evaluated based on the correlation (301).

2. The computing device (500) according to claim 1, characterized in that, Each of the plurality of items (204a, 204b, 204c, 204d) includes the output of an item model (201), and the item set (204) includes the plurality of items (204a, 204b, 204c, 204d) generated by the item model (201) based on the same input (203).

3. The computing device (500) according to claim 1 or 2, characterized in that, The project rating (205a, 205b, 205c, 205d) includes one or more constituent elements, each constituent element including a characteristic of the project (204a, 204b, 204c, 204d); optionally, the project rating (205a, 205b, 205c, 205d) includes a composite score of the constituent elements; optionally, the constituent elements are based on one or more of the following aspects of the project: usefulness, practicality, security, timeliness, clarity, and factual accuracy.

4. The computing device (500) according to any one of the preceding claims, characterized in that, The item ratings (205a, 205b, 205c, 205d) are based on human preferences; optionally, the item ratings (205a, 205b, 205c, 205d) are determined by humans and / or the item ratings (205a, 205b, 205c, 205d) are determined by an item rating model.

5. The computing device (500) according to any one of the preceding claims, characterized in that, The ratings (303a, 303b, 303c, 303d) generated by the rating model include simulated probabilities of the items (204a, 204b, 204c, 204d) as determined by the rating model (302) based on human preferences.

6. The computing device (500) according to any one of the preceding claims, characterized in that, The device (500) is used for: Based on the item scores (205a, 205b, 205c, 205d), a subset of items (204a, 204b, 204c, 204d) from the item set (204) is selected. Only for a subset of the selected items (204a, 204b, 204c, 204d) in the item set (204), obtain the ratings (303a, 303b, 303c, 303d) generated by the rating model.

7. The computing device (500) according to any one of the preceding claims, characterized in that, The device (500) is used to associate the item scores (205a, 205b, 205c, 205d) with the scores (303a, 303b, 303c, 303d) generated by the scoring model based on one or more of the following: the product of the item scores (205a, 205b, 205c, 205d) and the scores (303a, 303b, 303c, 303d) generated by the scoring model; or a correlation function between the item scores (205a, 205b, 205c, 205d) and the scores (303a, 303b, 303c, 303d) generated by the scoring model, such as Pearson correlation or Spearman correlation.

8. The computing device (500) according to any one of the preceding claims, characterized in that, The device (500) is used for: When referring to dependent claim 2, the plurality of projects (204a, 204b, 204c, 204d) are generated using the project model (201); and / or The scoring model (302) is used to determine the scores (303a, 303b, 303c, 303d) generated by the scoring model based on the plurality of items (204a, 204b, 204c, 204d).

9. The computing device (500) according to any one of claims 2 to 8, characterized in that, The project scores (205a, 205b, 205c, 205d) are not determined by the scoring model (302).

10. The computing device (500) according to any one of the preceding claims, characterized in that, The device (500) is used to: for each of the plurality of items (204a, 204b, 204c, 204d) in the item set (204), obtain a plurality of item scores (205a, 205b, 205c, 205d), wherein each item score (205a, 205b, 205c, 205d) of the items (204a, 204b, 204c, 204d) in the item set (204) is determined by a different person.

11. The computing device (500) according to any one of claims 2 to 10, characterized in that, The device (500) is used to: perform the steps of any one of the preceding claims for a plurality of inputs (203) to the project model (201) in order to evaluate the scoring model (302) based on all of the plurality of inputs (203).

12. The computing device (500) according to any one of claims 2 to 11, characterized in that, The project model (201) includes a language model, the output items (204a, 204b, 204c, 204d) of the project model include text, and the rating generated by the rating model is a numeric rating or an ordinal rating; optionally, the rating model (302) includes a language model; optionally, the language model includes one or more of the following: a neural model, an artificial model, a statistical model, and a multimodal language model.

13. The computing device (500) according to any one of claims 2 to 11, characterized in that, The project model (201) includes one or more of the following: image generation model, sound generation model, video generation model and physical project generation model; optionally, the scoring model (302) includes one or more of the following: image model, sound model, video model and physical project model.

14. The computing device (500) according to any one of the preceding claims, characterized in that, The device (500) is used for: Output the evaluation of the scoring model (302); The scoring model (302) is trained using the evaluation of the scoring model (302); and / or Different models are trained using the evaluation of the scoring model (302).

15. A method (400) for evaluating a scoring model, characterized in that, The method (400) includes the following steps: For each item in the multiple items in the project set, obtain the project score (401). For each of the plurality of projects in the project set, obtain the score generated by the scoring model, the score generated by the scoring model being determined by the scoring model (402). Associate the project score with the score generated by the scoring model (403). The scoring model (404) is evaluated based on the correlation.

Images