Prediction result generation method and device, equipment, medium and program product

Abstract

The embodiment of the invention discloses a prediction result generation method and device, equipment, a medium and a program product. A specific embodiment of the method comprises the steps of obtaining target input information and a pre-training language model set; a model output information set for the target input information is generated by utilizing the pre-training language model set, and each piece of model output information comprises a model prediction result group and a model prediction probability group; according to the obtained model prediction probability set, performing prediction result fusion on each model prediction result in the model prediction result set to generate a fusion prediction result set and a corresponding fusion prediction probability set; and according to the fusion prediction probability set, generating at least one actual prediction result. The implementation mode is related to artificial intelligence, and at least one actual prediction result for the target input information can be accurately and efficiently generated by utilizing the pre-training language model set.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of computer technology, and more specifically to methods, apparatus, devices, media, and program products for generating prediction results. Background Technology

[0002] Currently, with the rapid development of artificial intelligence, pre-trained large language models are being used more and more widely. Similarly, to save computational costs, hybrid expert large models have been developed. The common approach for predicting the input content of a model is to feed the input content into the hybrid expert large model to generate the corresponding output content.

[0003] However, the inventors discovered that when using the above method to generate output content, the following technical problems often occur:

[0004] The lack of differentiation among the expert networks in a hybrid expert model leads to inaccurate output. Furthermore, hybrid expert models suffer from complex training and slow convergence, resulting in high training costs.

[0005] The information disclosed in this background section is only intended to enhance the understanding of the background of the inventive concept, and therefore may contain information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0007] Some embodiments of this disclosure provide methods, apparatus, devices, media, and program products for generating prediction results to address one or more of the technical problems mentioned in the background section above.

[0008] In a first aspect, some embodiments of this disclosure provide a method for generating prediction results, including: acquiring target input information and a pre-trained language model set; using the pre-trained language model set to generate a model output information set for the target input information, wherein each model output information set includes: a model prediction result set and a model prediction probability set; fusing the prediction results of each model prediction result set in the model prediction result set according to the obtained model prediction probability set to generate a fused prediction result set and a corresponding fused prediction probability set; and generating at least one actual prediction result according to the fused prediction probability set, wherein the at least one actual prediction result is at least one fused prediction result in the fused prediction result set.

[0009] Optionally, the above-mentioned method of fusing prediction results of each model prediction result in the model prediction result set to generate a fused prediction result set and a corresponding fused prediction probability set, based on the obtained model prediction probability set, includes: in response to determining that there is a result inclusion relationship between at least two model prediction results, determining at least one target model prediction result set from the model prediction result set, wherein there is a result inclusion relationship between the model prediction results in the target model prediction result set; for each target model prediction result set in the at least one target model prediction result set, performing the following generation steps: fusing the prediction results of each target model prediction result in the target model prediction result set to obtain a fused prediction result; fusing the probability of each model prediction in the target model prediction probability set to generate an initial fused prediction probability, wherein the target model prediction probability set is a model prediction probability set corresponding to the target model prediction result set; generating the fused prediction probability set based on the obtained at least one initial fused prediction probability; and determining the obtained at least one fused prediction result as the fused prediction result set.

[0010] Optionally, generating the aforementioned fusion prediction probability set based on at least one obtained initial fusion prediction probability includes: removing each model prediction result corresponding to the at least one target model prediction result set from the aforementioned model prediction result set to obtain a removed model prediction result set; performing probability normalization on the at least one initial fusion prediction probability and the removed model prediction result set to obtain a normalized prediction probability set, which serves as the aforementioned fusion prediction probability set.

[0011] Optionally, the above-mentioned method of generating a model output information set for the target input information using the pre-trained language model set includes: in response to receiving the first computational value requirement information, inputting the target input information into each pre-trained language model in the pre-trained language model set to generate model output information and obtain the model output information set, wherein the first computational value requirement information indicates that the computational value consumption value is greater than or equal to the target value.

[0012] Optionally, the aforementioned pre-trained language model set is each expert model in a pre-trained hybrid expert large model; and the aforementioned method of generating a model output information set for the target input information using the aforementioned pre-trained language model set includes: in response to receiving second computational value requirement information, inputting the target input information into the aforementioned hybrid expert large model to generate a model output information set for a subset of pre-trained language models, wherein the aforementioned first computational value requirement information indicates that the computational value consumption value is less than the target value.

[0013] Optionally, the aforementioned pre-trained language model set consists of individual expert models in a hybrid expert large model; and the aforementioned method further includes: in response to receiving the second computational value requirement information, inputting the aforementioned target input information into the aforementioned hybrid expert large model to generate the aforementioned at least one actual prediction result.

[0014] Secondly, some embodiments of this disclosure provide a prediction result generation apparatus, comprising: an acquisition unit configured to acquire target input information and a pre-trained language model set; a first generation unit configured to generate a model output information set for the target input information using the pre-trained language model set, wherein each model output information set includes: a model prediction result set and a model prediction probability set; a fusion unit configured to fuse the prediction results of each model in the model prediction result set according to the obtained model prediction probability set, to generate a fused prediction result set and a corresponding fused prediction probability set; and a second generation unit configured to generate at least one actual prediction result according to the fused prediction probability set, wherein the at least one actual prediction result is at least one fused prediction result in the fused prediction result set.

[0015] Optionally, the first generation unit can be configured to: in response to determining that there is a result inclusion relationship between at least two model prediction results, determine at least one target model prediction result group from the set of model prediction result groups, wherein there is a result inclusion relationship between the model prediction results in the target model prediction result group; for each target model prediction result group in the at least one target model prediction result group, perform the following generation steps: fuse the prediction results of each target model prediction result in the target model prediction result group to obtain a fused prediction result; fuse the probabilities of each model prediction probability in the target model prediction probability group to generate an initial fused prediction probability, wherein the target model prediction probability group is a model prediction probability group corresponding to the target model prediction result group; generate the fused prediction probability set based on the obtained at least one initial fused prediction probability; and determine the obtained at least one fused prediction result as the fused prediction result set.

[0016] Optionally, the first generation unit can be configured to: remove each model prediction result corresponding to at least one target model prediction result group from the above model prediction result set to obtain a removed model prediction result set; perform probability normalization on the above at least one initial fusion prediction probability and the removed model prediction result set to obtain a normalized prediction probability set, which is used as the above fusion prediction probability set.

[0017] Optionally, the first generation unit can be configured to: in response to receiving the first computational value requirement information, input the target input information into each of the pre-trained language models in the pre-trained language model set to generate model output information and obtain the model output information set, wherein the first computational value requirement information indicates that the computational value consumption value is greater than or equal to the target value.

[0018] Optionally, the aforementioned pre-trained language model set consists of individual expert models in a pre-trained hybrid expert large model; and the first generation unit can be configured to: in response to receiving the second computational value requirement information, input the aforementioned target input information into the aforementioned hybrid expert large model to generate a model output information set for a subset of pre-trained language models, wherein the aforementioned second computational value requirement information indicates that the computational value consumption value is less than the target value.

[0019] Optionally, the aforementioned pre-trained language model set comprises various expert models in a hybrid expert large model; and the apparatus further includes: in response to receiving the second computational value requirement information, inputting the aforementioned target input information into the aforementioned hybrid expert large model to generate the aforementioned at least one actual prediction result.

[0020] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, such that when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any implementation of the first aspect.

[0021] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method as described in any implementation of the first aspect.

[0022] Fifthly, some embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the method described in any of the implementations of the first aspect above.

[0023] The above embodiments of this disclosure have the following beneficial effects: By using the prediction result generation method of some embodiments of this disclosure, and utilizing the pre-trained language model set, at least one actual prediction result for the target input information can be generated accurately and efficiently. Specifically, the reason for the inaccuracy of the relevant actual prediction results is that the differences between the various expert networks in the hybrid expert large model are insufficient, resulting in inaccurate final output content. In addition, the hybrid expert large model suffers from complex training and slow convergence, leading to high model training costs. Based on this, the prediction result generation method of some embodiments of this disclosure first obtains the target input information and the pre-trained language model set for use in generating at least one subsequent actual prediction result. Then, using the aforementioned pre-trained language model set, a model output information set for the aforementioned target input information can be accurately generated. Each model output information set includes: a model prediction result group and a model prediction probability group. Here, there are certain differences between the various expert networks in the pre-trained language model set, which can effectively avoid the problem of insufficient differences between the various expert networks in the hybrid expert large model, leading to inaccurate final output content. Furthermore, each pre-trained language model is pre-trained independently, avoiding the problems of complex training and slow convergence inherent in the hybrid expert large model. Therefore, using a pre-trained language model set, an accurate model output information set can be generated. Then, based on the obtained model prediction probability set, the prediction results of each model in the model prediction result set are fused to generate a fused prediction result set and a corresponding fused prediction probability set. Here, by performing prediction result fusion, a more accurate prediction result with unified output content and a corresponding fused prediction probability can be generated. Finally, based on the above fused prediction probability set, at least one actual prediction result can be accurately generated, wherein the at least one actual prediction result is at least one fused prediction result in the above fused prediction result set. In summary, an accurate model output information set can be generated using a pre-trained language model set. Based on this, by fusing the prediction results and corresponding prediction probabilities, the output content can be unified, facilitating the accurate generation of at least one actual prediction result. Attached Figure Description

[0024] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0025] Figure 1 This is a schematic diagram of an application scenario of a prediction result generation method according to some embodiments of the present disclosure;

[0026] Figure 2 This is a flowchart of some embodiments of the prediction result generation method based on this disclosure;

[0027] Figure 3 These are flowcharts of some other embodiments of the prediction result generation method according to this disclosure;

[0028] Figure 4 This is a flowchart of yet another embodiment of the prediction result generation method based on this disclosure;

[0029] Figure 5 These are schematic diagrams of some embodiments of the apparatus for generating prediction results based on this disclosure;

[0030] Figure 6 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0031] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0032] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0033] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0034] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0035] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0036] Before performing any of the operations involving the collection, storage, or use of user personal information (such as target input information) disclosed in this disclosure, the relevant organizations or individuals shall fulfill their obligations, including conducting personal information security impact assessments, informing personal information subjects, and obtaining prior authorization and consent from personal information subjects.

[0037] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0038] Figure 1 This is a schematic diagram of an application scenario of a prediction result generation method according to some embodiments of the present disclosure.

[0039] exist Figure 1In this application scenario, firstly, the electronic device 101 can acquire target input information 102 and a pre-trained language model set 103. In this application scenario, the target input information 102 can be "input information A". The pre-trained language model set 103 can include: pre-trained language model A1031, pre-trained language model B1032, pre-trained language model C1033, and pre-trained language model D1034. Then, the electronic device 101 can use the above-mentioned pre-trained language model set 103 to generate a model output information set 104 for the target input information 102. Each model output information includes: a model prediction result set and a model prediction probability set. In this application scenario, the model output information set 104 includes: model output information 1041, model output information 1042, model output information 1043, and model output information 1044. The model prediction result set corresponding to model output information 1041 includes: model prediction result 10411. Model output information 1041 corresponds to the model prediction probability group including: model prediction probability 10412. Model prediction result 10411 can be model prediction result A. Model prediction probability 10412 can be 70%. Model output information 1042 corresponds to the model prediction result group including: model prediction result 10421. Model output information 1042 corresponds to the model prediction probability group including: model prediction probability 10422. Model prediction result 10421 can be model prediction result B. Model prediction probability 10422 can be 80%. Model output information 1043 corresponds to the model prediction result group including: model prediction result 10431. Model output information 1043 corresponds to the model prediction probability group including: model prediction probability 10432. Model prediction result 10431 can be model prediction result C. Model prediction probability 10432 can be 77%. Model output information 1044 corresponds to the model prediction result group including: model prediction result 10441. The model output information 1044 corresponds to the model prediction probability set, which includes model prediction probability 10442. Model prediction result 10441 can be model prediction result D. Model prediction probability 10442 can be 82%. Next, the electronic device 101 can fuse the prediction results of each model prediction result in the obtained model prediction probability set to generate a fused prediction result set and a corresponding fused prediction probability set. In this application scenario, the fused prediction result set can include model prediction result 10411 and fused prediction result 10521. Fusion prediction result 10521 can be fused prediction result B. The fused prediction probability set can include fused prediction probability 10511 corresponding to model prediction result 10411 and fused prediction probability 10512 corresponding to fused prediction result 10521. Fusion prediction probability 10511 can be 60%. Fusion prediction probability 10512 can be 40%.Finally, the electronic device 101 can generate at least one actual prediction result based on the aforementioned fused prediction probability set. The at least one actual prediction result is at least one fused prediction result from the aforementioned fused prediction result set. In this application scenario, the at least one actual prediction result includes: model prediction result 10411.

[0040] It should be noted that the aforementioned electronic device 101 can be either hardware or software. When the electronic device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or as a single server or a single terminal device. When the electronic device is software, it can be installed in the hardware devices listed above. It can be implemented as, for example, multiple software programs or software modules used to provide distributed services, or as a single software program or software module. No specific limitations are made here.

[0041] It should be understood that Figure 1 The number of electronic devices shown is merely illustrative. Any number of electronic devices can be used depending on the implementation requirements.

[0042] Continue to refer to Figure 2 The diagram illustrates a flow 200 of some embodiments of a prediction result generation method according to the present disclosure. This prediction result generation method includes the following steps:

[0043] Step 201: Obtain target input information and pre-trained language model set.

[0044] In some embodiments, the execution entity of the above prediction result generation method (e.g. Figure 1 The electronic device 101 shown can acquire target input information and a pre-trained language model set via a wired or wireless connection. The target input information can be information to be predicted for subsequent output. For example, the target input information can be "keywords of the target article". The target input information can be text in prompt format. The pre-trained language models in the pre-trained language model set can be large, pre-trained language models. In practice, the pre-trained language model set can include, but is not limited to, at least one of the following: chatglm2-13b model, bai**an2-13b-chat model, qwen-7b-chat model. The number of pre-trained language models in the pre-trained language model set can be predetermined.

[0045] Step 202: Using the above-mentioned pre-trained language model set, generate a model output information set for the above-mentioned target input information.

[0046] In some embodiments, the executing entity can utilize the pre-trained language model set to generate a model output information set for the target input information. The model output information can be the output results of the model. Each model output information set includes: a model prediction result set and a model prediction probability set. The model prediction result set can contain at least one model prediction result. The model prediction probability set can contain at least one model prediction probability. The model prediction result can be the prediction result output by the model. For example, the model prediction result can be "keyword A, keyword B, keyword C, and keyword D". Each model prediction result in the model prediction result set can be the output result of a different pre-trained speech model. The number of model output information sets corresponding to the model output information set is less than or equal to the number of pre-trained language models corresponding to the pre-trained language model set. There is a one-to-one correspondence between the model prediction results in the model prediction result set and the model prediction probabilities in the model prediction probability set. The model prediction probability characterizes the accuracy of the model prediction result. The higher the model prediction probability, the more accurate the corresponding generated model prediction result.

[0047] As an example, the aforementioned execution entity can directly input the target input information into each pre-trained language model in the pre-trained language model set to generate a model output information set.

[0048] Step 203: Based on the obtained model prediction probability set, fuse the prediction results of each model in the model prediction result set to generate a fused prediction result set and a corresponding fused prediction probability set.

[0049] In some embodiments, the execution entity can fuse the prediction results of each model in the model prediction result set based on the obtained model prediction probability set to generate a fused prediction result set and a corresponding fused prediction probability set. There is a one-to-one correspondence between the fused prediction results in the fused prediction result set and the fused prediction probabilities in the fused prediction probability set. The fused prediction probability can characterize the accuracy of the fused prediction result. The fused prediction result can be a prediction result generated based on at least one model prediction result. The accuracy of the fused prediction result is higher than that of the corresponding at least one model prediction result.

[0050] As an example, firstly, the aforementioned execution entity can group the prediction results of each model in the model prediction result set according to similar result content to obtain various prediction result groups. Then, the prediction results in each prediction result group are semantically combined to generate a combined result, which serves as the fusion prediction result. Additionally, the prediction probabilities of each model in each corresponding model prediction probability group are averaged to generate an average probability, which serves as the fusion prediction probability, thus obtaining the fusion prediction result set and the corresponding fusion prediction probability set.

[0051] In some optional implementations of certain embodiments, the above-mentioned method of fusing the prediction results of each model in the model prediction result set to generate a fused prediction result set and a corresponding fused prediction probability set may include the following steps:

[0052] The first step involves determining, in response to the identification of a result inclusion relationship between at least two model prediction results, at least one target model prediction result group from the aforementioned set of model prediction result groups. Specifically, there is a result inclusion relationship between the model prediction results within the target model prediction result group. A result inclusion relationship characterizes a range inclusion relationship between the corresponding ranges of the model prediction results. The range of the results can be the semantic coverage that the prediction results can represent.

[0053] For example, the model prediction result group includes: the first model prediction result for "tomorrow" and the second model prediction result for "tomorrow". The range of the result corresponding to the first model prediction result is larger than the range of the result corresponding to the second model prediction result. That is, there is a range inclusion relationship between the first model prediction result and the second model prediction result.

[0054] As an example, the aforementioned execution entity can utilize a semantic recognition model to determine whether there exists a result inclusion relationship between at least two model prediction results.

[0055] The second step involves performing the following generation steps for each target model prediction result group in at least one of the target model prediction result groups mentioned above:

[0056] Sub-step 1 involves fusing the prediction results of each target model in the aforementioned target model prediction result group to obtain a fused prediction result. The range of the fused prediction result is greater than or equal to the range of the prediction results of each individual target model. In other words, the semantic range of the fused prediction result is greater than the semantic range of the prediction results of each individual target model.

[0057] As an example, firstly, the aforementioned execution entity can determine the semantic meaning of the prediction results corresponding to each prediction result of the target model in the target model prediction result group, thus obtaining a semantic group of prediction results. Then, the semantic group of prediction results is input into a text generation model to generate a fused prediction result. In practice, the text generation model can be a Transformer model.

[0058] Sub-step 2 involves fusing the prediction probabilities of each model in the target model prediction probability group to generate an initial fused prediction probability. Here, the target model prediction probability group is the model prediction probability group corresponding to the target model prediction result group.

[0059] As an example, the aforementioned execution entity can perform a weighted summation of the prediction probabilities of each model in the target model prediction probability group to generate an initial fusion prediction probability.

[0060] The third step is to generate the above-mentioned fusion prediction probability set based on at least one initial fusion prediction probability obtained.

[0061] As an example, the aforementioned executing entity may determine at least one initial fusion prediction probability as the aforementioned fusion prediction probability set.

[0062] Fourth, the aforementioned executing entity can determine at least one fusion prediction result as the aforementioned fusion prediction result set.

[0063] Optionally, generating the fusion prediction probability set based on at least one obtained initial fusion prediction probability may include the following steps:

[0064] The first step is for the execution entity to remove each model prediction result corresponding to at least one target model prediction result group from the model prediction result set to obtain the model prediction result set after removal.

[0065] The second step is that the aforementioned execution entity can perform probability normalization on the at least one initial fusion prediction probability and the predicted result set of the removed model to obtain a normalized prediction probability set, which is used as the aforementioned fusion prediction probability set.

[0066] As an example, the aforementioned execution entity can combine the at least one initial fusion prediction probability with the aforementioned removed model prediction result set to generate a probability set. Then, the probabilities in the probability set are normalized to generate a normalized prediction probability set, which serves as the aforementioned fusion prediction probability set.

[0067] Step 204: Generate at least one actual prediction result based on the above-mentioned fused prediction probability set.

[0068] In some embodiments, the executing entity may generate at least one actual prediction result based on the aforementioned fused prediction probability set. The actual prediction result is the final, accurate prediction result. The number of prediction results corresponding to at least one actual prediction result may be preset. For example, the number of prediction results corresponding to at least one actual prediction result may be 20. The aforementioned at least one actual prediction result is at least one fused prediction result from the aforementioned fused prediction result set.

[0069] As an example, the aforementioned execution entity can filter out the fusion prediction results whose corresponding fusion prediction probability values ​​are among the top prediction results from the fusion prediction result set, and use them as actual prediction results to obtain at least one actual prediction result.

[0070] In some alternative implementations of certain embodiments, the aforementioned pre-trained language model set may be the individual expert models within a mixed expert large model. For example, the pre-trained language model set may be the individual expert models within a Mixtral-8x7B MoE model.

[0071] Optionally, in response to receiving the second computational value requirement information, the aforementioned target input information is input into the aforementioned hybrid expert large model to generate at least one actual prediction result. Wherein, at least one actual prediction result corresponds to at least one expert model among the various expert models.

[0072] The above embodiments of this disclosure have the following beneficial effects: By using the prediction result generation method of some embodiments of this disclosure, and utilizing the pre-trained language model set, at least one actual prediction result for the target input information can be generated accurately and efficiently. Specifically, the reason for the inaccuracy of the relevant actual prediction results is that the differences between the various expert networks in the hybrid expert large model are insufficient, resulting in inaccurate final output content. In addition, the hybrid expert large model suffers from complex training and slow convergence, leading to high model training costs. Based on this, the prediction result generation method of some embodiments of this disclosure first obtains the target input information and the pre-trained language model set for use in generating at least one subsequent actual prediction result. Then, using the aforementioned pre-trained language model set, a model output information set for the aforementioned target input information can be accurately generated. Each model output information set includes: a model prediction result group and a model prediction probability group. Here, there are certain differences between the various expert networks in the pre-trained language model set, which can effectively avoid the problem of insufficient differences between the various expert networks in the hybrid expert large model, leading to inaccurate final output content. Furthermore, each pre-trained language model is pre-trained independently, avoiding the problems of complex training and slow convergence inherent in the hybrid expert large model. Therefore, using a pre-trained language model set, an accurate model output information set can be generated. Then, based on the obtained model prediction probability set, the prediction results of each model in the model prediction result set are fused to generate a fused prediction result set and a corresponding fused prediction probability set. Here, by performing prediction result fusion, a more accurate prediction result with unified output content and a corresponding fused prediction probability can be generated. Finally, based on the above fused prediction probability set, at least one actual prediction result can be accurately generated, wherein the at least one actual prediction result is at least one fused prediction result in the above fused prediction result set. In summary, an accurate model output information set can be generated using a pre-trained language model set. Based on this, by fusing the prediction results and corresponding prediction probabilities, the output content can be unified, facilitating the accurate generation of at least one actual prediction result.

[0073] Further reference Figure 3 The diagram illustrates a flow 300 of another embodiment of the prediction result generation method according to the present disclosure. This prediction result generation method includes the following steps:

[0074] Step 301: Obtain target input information and pre-trained language model set.

[0075] Step 302: In response to receiving the first computational value requirement information, the target input information is input into each pre-trained language model in the pre-trained language model set to generate model output information, thereby obtaining the model output information set.

[0076] In some embodiments, in response to receiving a first computational value request information, the executing entity (e.g. Figure 1 The electronic device shown can input the aforementioned target input information into each pre-trained language model in the aforementioned pre-trained language model set to generate model output information, thus obtaining the aforementioned model output information set. The aforementioned first computational value requirement information represents that the computational value consumption value is greater than or equal to the target value. In practice, the computational value consumption value can be the model's computational cost. The target value can be a pre-set value. The target value can be determined based on the number of model parameters in the pre-trained language model set. Each pre-trained language model in the pre-trained language model set can be pre-trained independently.

[0077] Step 303: Based on the obtained model prediction probability set, fuse the prediction results of each model in the model prediction result set to generate a fused prediction result set and a corresponding fused prediction probability set.

[0078] Step 304: Generate at least one actual prediction result based on the above-mentioned fused prediction probability set.

[0079] In some embodiments, the specific implementation of steps 301 and 303-304 and their resulting technical effects can be found in [reference needed]. Figure 2 Steps 201 and 203-204 in the corresponding embodiments will not be repeated here.

[0080] from Figure 3 It can be seen from this that, with Figure 2 Compared to the description of some corresponding embodiments, Figure 3 In some corresponding embodiments, the prediction result generation method process 300, under the first computational value requirement information, can accurately generate the model output information set by utilizing a set of pre-trained language models with parallel structural differences.

[0081] Further reference Figure 4 The diagram illustrates a flow 400 of yet another embodiment of the prediction result generation method according to the present disclosure. This prediction result generation method includes the following steps:

[0082] Step 401: Obtain target input information and pre-trained language model set.

[0083] Step 402: In response to receiving the second computational value requirement information, the target input information is input into the hybrid expert large model to generate a model output information set for a subset of pre-trained language models.

[0084] In some embodiments, in response to receiving second computational value requirement information, the aforementioned target input information is input into the aforementioned hybrid expert large model, and the executing entity (e.g., Figure 1 The electronic device shown can input the aforementioned target input information into the aforementioned hybrid expert large model to generate a model output information set for a subset of pre-trained language models. The aforementioned second computational value requirement requires that the computational value consumption of the information representation be less than the target value.

[0085] The hybrid expert model comprises a routing layer (gateway layer) and attention layers within each expert model. That is, the output of the hybrid expert model is the output of at least one expert model selected by the routing layer. There is a one-to-one correspondence between the pre-trained language models in the pre-trained language model subset. The pre-trained language model subset can be selected from the pre-trained language model set based on the routing layer.

[0086] Step 403: Based on the obtained model prediction probability set, fuse the prediction results of each model in the model prediction result set to generate a fused prediction result set and a corresponding fused prediction probability set.

[0087] Step 404: Generate at least one actual prediction result based on the above-mentioned fused prediction probability set.

[0088] In some embodiments, the specific implementation of steps 401 and 403-404 and their resulting technical effects can be found in [reference needed]. Figure 2 Steps 201 and 203-204 in the corresponding embodiments will not be repeated here.

[0089] from Figure 4 It can be seen from this that, with Figure 2 Compared to the description of some corresponding embodiments, Figure 4 In some corresponding embodiments, the prediction result generation method process 400, under the second computational value requirement information, can accurately generate the model output information set by utilizing the pre-trained language model set in the hybrid expert large model.

[0090] Further reference Figure 5 As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of a prediction result generation apparatus, which are similar to... Figure 2 Corresponding to the method embodiments shown, this prediction result generation device can be specifically applied to various electronic devices.

[0091] like Figure 5 As shown, a prediction result generation device 500 includes: an acquisition unit 501, a first generation unit 502, a fusion unit 503, and a second generation unit 504. The acquisition unit 501 is configured to acquire target input information and a pre-trained language model set; the first generation unit 502 is configured to generate a model output information set for the target input information using the pre-trained language model set, wherein each model output information set includes a model prediction result set and a model prediction probability set; the fusion unit 503 is configured to fuse the prediction results of each model in the model prediction result set according to the obtained model prediction probability set, to generate a fused prediction result set and a corresponding fused prediction probability set; the second generation unit 504 is configured to generate at least one actual prediction result based on the fused prediction probability set, wherein the at least one actual prediction result is at least one fused prediction result in the fused prediction result set.

[0092] In some optional implementations of certain embodiments, the fusion unit 503 may be further configured to: in response to determining that there is a result inclusion relationship between at least two model prediction results, determine at least one target model prediction result group from the set of model prediction result groups, wherein there is a result inclusion relationship between the model prediction results in the target model prediction result group; for each target model prediction result group in the at least one target model prediction result group, perform the following generation steps: fuse the prediction results of each target model prediction result in the target model prediction result group to obtain a fused prediction result; fuse the probabilities of each model prediction probability in the target model prediction probability group to generate an initial fused prediction probability, wherein the target model prediction probability group is a model prediction probability group corresponding to the target model prediction result group; generate the fused prediction probability set based on the obtained at least one initial fused prediction probability; and determine the obtained at least one fused prediction result as the fused prediction result set.

[0093] In some optional implementations of some embodiments, the fusion unit 503 may be further configured to: remove each model prediction result corresponding to the at least one target model prediction result group from the model prediction result group set to obtain a removed model prediction result set; perform probability normalization on the at least one initial fusion prediction probability and the removed model prediction result set to obtain a normalized prediction probability set as the fusion prediction probability set.

[0094] In some optional implementations of some embodiments, the first generation unit 502 may be further configured to: in response to receiving the first computational value requirement information, input the target input information into each pre-trained language model in the pre-trained language model set to generate model output information and obtain the model output information set, wherein the first computational value requirement information indicates that the computational value consumption value is greater than or equal to the target value.

[0095] In some optional implementations of some embodiments, the aforementioned pre-trained language model set is each expert model in a pre-trained hybrid expert large model; and the first generation unit 502 may be further configured to: in response to receiving the second computational value requirement information, input the aforementioned target input information into the aforementioned hybrid expert large model to generate a model output information set for a subset of pre-trained language models, wherein the aforementioned second computational value requirement information indicates that the computational value consumption value is less than the target value.

[0096] In some alternative implementations of some embodiments, the aforementioned pre-trained language model set is each expert model in a pre-trained hybrid expert large model; and the first generation unit 502 may be further configured to: in response to receiving the second computational value requirement information, input the aforementioned target input information into the aforementioned hybrid expert large model to generate the aforementioned at least one actual prediction result.

[0097] It is understandable that the units described in the prediction result generation device 500 and the reference Figure 2 The steps in the described method correspond to each other. Therefore, the operations, features, and beneficial effects described above for the method also apply to the prediction result generation device 500 and the units contained therein, and will not be repeated here.

[0098] The following is for reference. Figure 6 It illustrates electronic devices suitable for implementing some embodiments of this disclosure (e.g., Figure 1 A schematic diagram of the structure of electronic device 101)600 in the middle. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0099] like Figure 6As shown, the electronic device 600 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory 602 or a program loaded from a storage device 608 into a random access memory 603. The random access memory 603 also stores various programs and data required for the operation of the electronic device 600. The processing unit 601, the read-only memory 602, and the random access memory 603 are interconnected via a bus 604. An input / output interface 605 is also connected to the bus 604.

[0100] Typically, the following devices can be connected to the input / output interface 605: input devices 606 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 607 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 608 including, for example, magnetic tape, hard disk, etc.; and communication devices 609. Communication device 609 allows electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 An electronic device 600 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 6 Each box shown can represent a device or multiple devices as needed.

[0101] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a storage device 608, or installed from a read-only memory 602. When the computer program is executed by the processing device 601, it performs the functions defined above in the methods of some embodiments of this disclosure.

[0102] It should be noted that, in some embodiments of this disclosure, the computer-readable medium described above may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0103] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0104] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: acquire target input information and a pre-trained language model set; generate a set of model output information for the target input information using the pre-trained language model set, wherein each model output information includes: a set of model prediction results and a set of model prediction probabilities; fuse the prediction results of each model prediction result in the obtained set of model prediction probabilities to generate a fused prediction result set and a corresponding fused prediction probability set; and generate at least one actual prediction result based on the fused prediction probability set, wherein the at least one actual prediction result is at least one fused prediction result in the fused prediction result set.

[0105] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0106] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0107] The units described in some embodiments of this disclosure can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor may be described as including an acquisition unit, a first generation unit, a fusion unit, and a second generation unit. The names of these units do not necessarily limit the specific unit; for example, the acquisition unit may also be described as "a unit that acquires target input information and a pre-trained language model set".

[0108] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0109] Some embodiments of this disclosure also provide a computer program product, including a computer program that, when executed by a processor, implements any of the above-described prediction result generation methods.

[0110] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A method for generating prediction results, comprising: Obtain target input information and a pre-trained language model set; Using the pre-trained language model set, a model output information set is generated for the target input information, wherein each model output information set includes: a model prediction result set and a model prediction probability set; Based on the obtained model prediction probability set, the prediction results of each model in the model prediction result set are fused to generate a fused prediction result set and a corresponding fused prediction probability set. Based on the fusion prediction probability set, at least one actual prediction result is generated, wherein the at least one actual prediction result is at least one fusion prediction result in the fusion prediction result set.

2. The method according to claim 1, wherein, The step of fusing the prediction results of each model in the model prediction result set based on the obtained model prediction probability set to generate a fused prediction result set and a corresponding fused prediction probability set includes: In response to determining that there is a result inclusion relationship between at least two model prediction results, at least one target model prediction result group is determined from the set of model prediction result groups, wherein there is a result inclusion relationship between the model prediction results in the target model prediction result group; For each target model prediction result group in the at least one target model prediction result group, perform the following generation steps: The prediction results of each target model in the target model prediction result group are fused to obtain the fused prediction result; The prediction probabilities of each model in the target model prediction probability group are fused to generate an initial fused prediction probability, wherein the target model prediction probability group is the model prediction probability group corresponding to the target model prediction result group. The fusion prediction probability set is generated based on at least one obtained initial fusion prediction probability; At least one of the obtained fusion prediction results is determined as the fusion prediction result set.

3. The method according to claim 2, wherein, The step of generating the fusion prediction probability set based on at least one obtained initial fusion prediction probability includes: Remove each model prediction result corresponding to the at least one target model prediction result group from the model prediction result set to obtain the removed model prediction result set. Perform probability normalization on the at least one initial fusion prediction probability and the removed model prediction result set to obtain a normalized prediction probability set, which is used as the fusion prediction probability set.

4. The method according to claim 1, wherein, The step of generating a model output information set for the target input information using the pre-trained language model set includes: In response to receiving the first computational value requirement information, the target input information is input to each pre-trained language model in the pre-trained language model set to generate model output information, thereby obtaining the model output information set, wherein the first computational value requirement information indicates that the computational value consumption value is greater than or equal to the target value.

5. The method according to claim 1, wherein, The pre-trained language model set consists of individual expert models from a pre-trained hybrid expert large model. as well as The step of generating a model output information set for the target input information using the pre-trained language model set includes: In response to receiving the second computational value requirement information, the target input information is input into the hybrid expert large model to generate a model output information set for a subset of pre-trained language models, wherein the second computational value requirement information indicates that the computational value consumption value is less than the target value.

6. The method according to claim 4, wherein, The pre-trained language model set consists of individual expert models within the hybrid expert large model; as well as The method further includes: In response to receiving the second computational value requirement information, the target input information is input into the hybrid expert large model to generate the at least one actual prediction result.

7. A prediction result generation apparatus, comprising: The acquisition unit is configured to acquire target input information and a pre-trained language model set; The first generation unit is configured to generate a set of model output information for the target input information using the pre-trained language model set, wherein each set of model output information includes: a set of model prediction results and a set of model prediction probabilities. The fusion unit is configured to fuse the prediction results of each model in the model prediction result set according to the obtained model prediction probability set, so as to generate a fused prediction result set and a corresponding fused prediction probability set. The second generation unit is configured to generate at least one actual prediction result based on the fusion prediction probability set, wherein the at least one actual prediction result is at least one fusion prediction result in the fusion prediction result set.

8. An electronic device, comprising: One or more processors; Storage device, on which one or more programs are stored, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-6.

9. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-6.

10. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-6.

Images