Large model correction method

Abstract

A large model correction method. The method comprises: acquiring reference large model data and / or large model data to be corrected, wherein the reference large model data comprises a first learning dataset and a first algorithm, and the large model data to be corrected comprises a second learning dataset and a second algorithm (101); on the basis of at least one or more of the following correction modes, performing a correction operation on a large model to be corrected: correcting the second algorithm on the basis of the first learning dataset, correcting the quality of the first learning dataset as reference model input data on the basis of the second algorithm, correcting the second algorithm on the basis of the first learning dataset and the second learning dataset, and correcting the quality of the second learning dataset on the basis of the second algorithm and the first learning dataset (102); and on the basis of at least one or more correction operations, obtaining a corrected large model (103). On the basis of the comparison effect of a reference large model, a corrected large model is enhanced in terms of both datasets and algorithms, providing good model support for specific downstream applications.

Description

A large model correction method

[0001] This application claims priority to the Chinese patent application No. CN 202410434925.0, filed on April 10, 2024, entitled “A large model correction method”, the disclosure of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the field of artificial intelligence, in particular, to a large model correction method. BACKGROUND

[0003] With the development of new generation information technology, artificial intelligence, big data and other technologies have been gradually applied to people's actual work, greatly improving people's work efficiency. In particular, the emergence of artificial intelligence large models in recent years, due to their ability to understand user intent, effective multi-round communication, and answer content, they have particularly outstanding performance in language understanding, text generation, and knowledge reasoning.

[0004] However, existing artificial intelligence large models have some problems. For example, large language models (LLMs) have large differences in answers, such as generating content that does not match user input, contradicts previously generated content, or does not match known world knowledge. Even if a large model is well evaluated, it may directly affect its application performance in downstream tasks after iteration or drift.

[0005] Therefore, there is an urgent need for a technical solution to correct large models, such as LLMs, to generate the most suitable prompts even with the most comprehensive data set and iterative evolution, thereby improving performance in downstream tasks. SUMMARY

[0006] The embodiments of the present application provide at least a large model correction method to achieve more comprehensive and accurate large model correction through various model correction methods, thereby improving the application performance in subsequent downstream tasks.

[0007] In a first aspect, the embodiments of the present application provide a large model correction method, comprising:

[0008] obtaining reference large model data and / or to-be-corrected large model data; the reference large model data includes a first learning data set and a first algorithm, and the to-be-corrected large model data includes a second learning data set and a second algorithm;

[0009] performing a correction operation on the to-be-corrected large model according to at least one or more of the following correction methods:

[0010] correcting the second algorithm based on the first learning data set, correcting quality of the first learning data set as reference model input data based on the second algorithm, correcting the second algorithm based on the first learning data set and the second learning data set, correcting quality of the second learning data set based on the second algorithm and the first learning data set;

[0011] based on at least one or more of the correction operations, obtaining a corrected large model.

[0012] Optionally, the reference large model data further comprises a first factor relationship table, the first factor relationship table being used to represent an association relationship between each first factor; and the large model data to be corrected further comprises a second factor relationship table, the second factor relationship table being used to represent an association relationship between each second factor.

[0013] Optionally, the correcting the second algorithm based on the first learning data set comprises:

[0014] labeling the first learning data set based on the first algorithm to obtain a labeled first learning data set;

[0015] performing calculation on the labeled first learning data set based on a simplified relationship corresponding to the second algorithm to determine an output factor relationship table, and determining whether the output factor relationship table appears in the first factor relationship table;

[0016] performing calculation on the labeled first learning data set based on the second algorithm to determine an output factor relationship table, and determining whether the output factor relationship table appears in the first factor relationship table;

[0017] determining whether a fitting module needs to be added for the second algorithm based on the determination result, the fitting module being used to fit the first factor relationship.

[0018] Optionally, after it is determined that the fitting module needs to be added for the second algorithm, the method further comprises:

[0019] determining a minimum amount of data required for fitting the first factor relationship based on the fitting module;

[0020] adjusting the first learning data set based on the minimum amount of data to obtain an adjusted first learning data set.

[0021] Optionally, the correcting quality of the first learning data set as reference model input data based on the second algorithm comprises:

[0022] inputting the unlabeled first learning data set into the second algorithm to perform calculation, and obtaining an output factor relationship table;

[0023] judging whether the output factor relationship table contains the first factor relationship in the first factor relationship table;

[0024] adjusting the quality of the first learning data set based on the judging result.

[0025] Optionally, the correcting the second algorithm based on the first learning data set and the second learning data set comprises:

[0026] mixing the first learning data set and the second learning data set to obtain a mixed learning data set;

[0027] performing calculation on the second learning data set and the mixed learning data set based on the simplified relationship corresponding to the second algorithm to determine an output factor relationship table;

[0028] judging whether the output factor relationship table contains the first factor relationship in the first factor relationship table;

[0029] determining whether the first learning data set and / or the second learning data set need to be added based on the judging result.

[0030] Optionally, the first learning data set and / or the second learning data set is added in at least one or more of the following ways:

[0031] selecting learning data to be added from the first learning data set;

[0032] estimating missing second learning data based on the first learning data set and the second algorithm;

[0033] performing format conversion or other operations on part of the learning data to obtain learning data suitable for mixed analysis of the first learning data set and the second learning data set.

[0034] Optionally, in the case that the learning data to be added is non-continuous data, the learning data to be added is selected from the first learning data set in one of the following ways:

[0035] sampling based on discrete distribution probability;

[0036] performing inverse operation on the second algorithm based on Markov chain, and sampling the operation result based on distribution probability.

[0037] Optionally, the correcting the quality of the second learning data set comprises:

[0038] mixing the first learning material set and the second learning material set to obtain a mixed learning material set;

[0039] inputting the mixed learning material set into the second algorithm to perform calculation, and determining an output factor relationship table;

[0040] determining whether the output factor relationship table appears the first factor relationship in the first factor relationship table;

[0041] adjusting the quality of the second learning material set based on the determination result.

[0042] Optionally, the adjusting the quality of the second learning material set based on the determination result comprises:

[0043] if the first factor relationship in the first factor relationship table does not appear, performing label annotation on the mixed learning material set composed of the second learning material set and the first learning material set by the second algorithm;

[0044] removing the second learning material whose specified factor label does not conform to the specified factor label distribution of the first learning material set from the second learning material set, and obtaining an updated mixed learning material set;

[0045] inputting the updated mixed learning material set into the second algorithm to perform calculation again, and determining whether the output factor relationship table appears the first factor relationship in the first factor relationship table;

[0046] if the first factor relationship in the first factor relationship table still does not appear, using a random sampling method or a stratified sampling method to screen out the corresponding second learning material to obtain an updated mixed learning material set with emerging new relationships or non-related relationships being uniform; wherein, the screened-out material set is reserved for subsequent large model iterative updating.

[0047] Using the above large model correction method, when the reference large model data and the to-be-corrected large model data are obtained, at least one or more correction operations of the to-be-corrected large model can be performed: the correction operations include but are not limited to correcting the second algorithm based on the first learning material set, correcting the quality of the first learning material set as reference model input data based on the second algorithm, correcting the second algorithm based on the first learning material set and the second learning material set, and correcting the quality of the second learning material set based on the second algorithm and the first learning material set. It can be seen that, based on the comparison of the reference large model, the corrected large model after correction has been greatly enhanced in terms of material set and algorithm, which provides good model support for subsequent specific applications, and is more practical.

[0048] Other advantages of the present application will be set forth in part in the description which follows, and in part will be obvious from the description, or can be learned by practice of the application.

[0049] It is to be understood that the above description is an overview of the technical solutions of the present application, so as to enable a general understanding of the technical means of the present application, and then the content of the specification is implemented. In order to make the above and other purposes, characteristics and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application are described below. BRIEF DESCRIPTION OF DRAWINGS

[0050] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the embodiments will be briefly introduced. The drawings incorporated into the specification and form part of the specification, which shows the embodiments consistent with the present application, and together with the specification for the description of the technical solutions of the present application. It should be understood that the drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope of protection, for those skilled in the art can obtain other related drawings without creative labor on the basis of these drawings. Moreover, in the whole drawings, the same reference signs represent the same parts. In the drawings:

[0051] Fig. 1 shows a flowchart of a large model correction method provided by an embodiment of the present application;

[0052] Fig. 2 shows a schematic diagram of a large model correction device provided by an embodiment of the present application;

[0053] Fig. 3 shows a schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0054] The exemplary embodiments of the present application will be described in detail with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms, and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided so that the present application can be more thoroughly understood, and the scope of the present application can be fully conveyed to those skilled in the art.

[0055] In the description of the embodiments of the present application, it should be understood that terms such as "include" or "have" are intended to indicate that the features, numbers, steps, actions, components, parts or combinations thereof disclosed in the specification exist, and do not exclude the possibility of existence of one or more other features, numbers, steps, actions, components, parts or combinations thereof.

[0056] Unless otherwise defined, “or” means any one member of a set or all, which is dependent from convenience regarding the description. “And / or” as used herein is to be interpreted as a conjunctive relationship, meaning that there are three possible relationships: A and / or B means that A exists alone, A and B exist together, or B exists alone.

[0057] The terms “first”, “second”, etc. are only used to distinguish the same or similar technical features for the convenience of description, and cannot be understood as indicating or implying the relative importance or quantity of the technical features. Therefore, the features defined by “first”, “second”, etc. can explicitly or implicitly include one or more of such features. In the description of the embodiments of the present application, unless otherwise specified, the meaning of the term “plurality” is two or more than two.

[0058] It is found through research that the related art lacks a correction method after the model is used, cannot adapt to model correction across models or across generations, and sometimes cannot solve the model drift generated after iteration. In short, there is no standard for large model correction.

[0059] Previously, during the model learning stage, the data was mainly selected manually, and the drift problem generated after the model went online and iterated had no solution. For example, the most advanced large language model “lazy” in the industry could not be solved for nearly a month, and the problems of generating content inconsistent with user input, contradictory to previously generated content, or inconsistent with known world knowledge limited the performance of the model.

[0060] To solve the above problems, some researchers use an additional closed model to perform AI alignment (regulation) on the output answer to confirm it again.

[0061] However, the above technical method needs to perform two calculations, which is inefficient and has poor prediction effect, and is limited by the closed model to form a bottleneck for the output of the entire large model, which may limit the advantages of large model iteration optimization.

[0062] To at least partially solve one or more of the above problems and other potential problems, the present application provides a more comprehensive and accurate large model correction scheme through various model correction methods, which directly corrects the large model and has great advantages in terms of calculation time and model iteration optimization. It can be applied to the learning stage and the correction stage after the large model goes online, and has more extensive applicability.

[0063] For the convenience of understanding the present embodiment, first, a large model correction method disclosed by the present application is introduced in detail. The execution subject of the large model correction method provided by the present application is generally an electronic device with certain computing power, which includes, for example, a server or other processing device. In some possible implementation manners, the large model correction method can be realized by a processor calling computer readable instructions stored in a memory.

[0064] Referring to FIG. 1, a flowchart of a large model correction method provided by the present application is shown, and the method includes the following steps S101-S103:

[0065] S101: Obtain reference large model data and / or to-be-corrected large model data; the reference large model data includes a first learning material set and a first algorithm, and the to-be-corrected large model data includes a second learning material set and a second algorithm;

[0066] S102: Perform a correction operation on the to-be-corrected large model according to at least one or more of the following correction methods:

[0067] correct the second algorithm based on the first learning material set, correct the quality of the first learning material set as reference model input data based on the second algorithm, correct the second algorithm based on the first learning material set and the second learning material set, and correct the quality of the second learning material set based on the second algorithm and the first learning material set;

[0068] S103: Obtain a corrected large model based on at least one or more correction operations.

[0069] In order to facilitate understanding of the model correction method provided by the present application, the application scenario of the method is first introduced. The model correction method provided by the present application can be mainly used in the field of artificial intelligence technology, for example, it can be applied to various artificial intelligence subfields such as image recognition and face detection, without specific limitation here. In particular, in the large model learning stage and the correction stage after going online, the corrected large model has better model performance, so as to adapt to more complex and variable downstream application scenarios.

[0070] The reference large model data herein can include one or more reference large models, and the large model to be corrected can also include one or more correction models. In actual applications, the embodiments of the present application can be used to correct different models with the same reference large model, correct a single model with different reference large models, or correct different models with different reference large models, such as Agent models, thereby forming Agents large models. Other various application scenarios are also possible, which are not specifically limited herein. Next, one reference large model and one large model to be corrected are exemplarily described, such as large model A as a correction template to correct large model B as a correction object.

[0071] For the reference large model data, it can include a first learning data set and a first algorithm. Similarly, for the large model to be corrected, it can include a second learning data set and a second algorithm.

[0072] Based on different target application scenarios, the learning data set obtained herein is also different. For example, for the application scenario of high-speed mobile safe driving, the learning data set can be related road images, and can also be other related detection data (such as sound or other time series data that can be collected by sensors). The form of the learning data and the combination manner thereof are not specifically limited herein. Next, image data is exemplarily described.

[0073] The algorithm includes but is not limited to a sampling algorithm. In actual applications related to terminals, the algorithm herein can be determined by the terminal based on certain application requirements. For example, when face recognition is needed in a rainy driving environment, it can be determined that what kind of algorithm needs to be used to extract the face recognition method from the image. Based on the corresponding algorithm, each learning data in the learning data set can be factorially labeled, that is, for example, the influence on safety can be labeled.

[0074] In the actual model correction process of the embodiments of the present application, multiple dimensions can be used to achieve different correction methods with their own correction points. For example, the first learning data set is used to correct the second algorithm (hereinafter referred to as correction I): the newly added module of the second algorithm is mainly corrected; the second algorithm is used to correct the quality of the first learning data set as reference model input data (hereinafter referred to as correction II): the quality of the first learning data set (minimum quantity, factor composition distribution anomaly, etc.) is mainly confirmed; the first learning data set and the second learning data set are used to correct the second algorithm (hereinafter referred to as correction III): the second algorithm is mainly corrected, and the convolution module and other modules are mainly corrected; the second algorithm and the first learning data set are used to correct the quality of the second learning data set (hereinafter referred to as correction IV): the quality of the second learning data set (sample composition deviation, etc.) is mainly corrected.

[0075] In practical applications, the above various correction methods can be partially omitted, and the correction order can be changed. For example, the correction can be performed in the order of correction I-correction II-correction III-correction IV, and the correction order can be adjusted or some correction methods can be omitted according to the correction situation. The basic direction is to make the correction algorithm compatible, adjust the data set template, and then supplement the missing data and adjust the large model data set.

[0076] To achieve better model correction performance, the reference large model data herein further includes a first factor relationship table for representing the association relationship between each first factor; and the large model data to be corrected further includes a second factor relationship table for representing the association relationship between each second factor.

[0077] The factor relationship table can be extracted from a large factor relationship table database, and mainly includes various factor items and the association relationship between various factor items. Taking the target application scenario of high-speed mobile safe driving as an example, the factor items can include “influence image”, “influence safety”, “humidity”, “color temperature”, and other image recognition influence factor items. The relationship between the factor items can be a correlation relationship, and can also be a logical relationship (function relationship, cause and effect relationship, coverage relationship, indirect relationship, parallel relationship, etc.).

[0078] In the embodiments of the present application, the first factor relationship in the first factor relationship table can be taken as a specified factor relationship, which can be the relationship (including parameters) of the specified factor and all related factors in the first factor relationship table.

[0079] The embodiments of the present application confirm whether the first learning data set of the reference large model and the first algorithm can calculate the factor relationship, and perform the second learning data set of the large model to be corrected and the second algorithm correction. Through the second algorithm, the algorithm can learn the second learning data set to generate stable and compatible labels of the reference large model, and also compatible with the addition of the data set. The algorithm iterates, so the model correction between cross-model or cross-generation can be performed, the iteration advantage of compatible data set addition is compatible, and the model drift problem is avoided.

[0080] Considering the key role of various correction methods in implementing model correction in the embodiments of the present application, different correction methods will be described in the following aspects.

[0081] Here, in order to facilitate the description of the specific implementation process of the above various correction methods, the following examples can be combined to illustrate.

[0082] Wherein, the reference large model is taken as a correction template, and the large model to be corrected is taken as a correction object, wherein the correction template is denoted as large model A, mainly including: data set A (i.e. first learning data set), algorithm A (i.e. first algorithm), factor table A (i.e. first factor relationship table), and the correction object is denoted as large model B, mainly including: data set B (i.e. second learning data set), algorithm B (i.e. second algorithm), and factor table B (i.e. second factor relationship table).

[0083] The first aspect is to correct the second algorithm based on the first learning data set, specifically including the following steps:

[0084] Step one, label annotation is performed on the first learning data set by the first algorithm to obtain a labeled first learning data set;

[0085] Step two, based on the simplified relationship corresponding to the second algorithm, the labeled first learning data set is calculated to determine the output factor relationship table, and it is judged whether the output factor relationship table appears in the first factor relationship in the first factor relationship table;

[0086] Step three, based on the second algorithm, the labeled first learning data set is calculated to determine the output factor relationship table, and it is judged whether the output factor relationship table appears in the first factor relationship in the first factor relationship table;

[0087] Step four, based on the judgment result, it is determined whether a fitting module needs to be added for the second algorithm, and the fitting module is used to fit the first factor relationship.

[0088] In combination with the above example, the recursive relationship here can be data set A—> algorithm B (specified "simplified relationship" algorithm)—> factor table BI, which is compared with the recursive relationship of data set A—> algorithm A—> factor table A. If factor table BI has a similar function of "specified factor relationship" selected by factor table A, or product roll, logical relationship, it can be confirmed that algorithm B is compatible with the underlying module of algorithm A.

[0089] As mentioned above, according to the statistical theorem and the assumption behind the large model, if the data set is the same and the algorithm is compatible, the same factor relationship will be calculated in principle.

[0090] In practical application, if a certain factor is specified in the large model, the factor relationship related to the same roll (or 2-layer inner convolution) can be cut off. For example, in the recognition of traffic lights, the factor relationship table of the picture factor of data set A traffic light is analyzed, and the factor relationship table is compared to determine a factor relationship table feature.

[0091] Then these characteristic factors can be used as specified factors, and the specified factor relationship is intercepted for correction, which mainly considers that the more the surface layer is convoluted, the more the data structure is different, such as whether the picture format is JPG or tif, and the more the bottom layer is convoluted, the more the characteristic attributes are, and there are also multiple ln(x) linear relationships as described above, and in some large models, the algorithm may be different, that is, one convolution layer is composed of multiple convolution layers.

[0092] After determining the specified factor relationship, the correction of the algorithm B can be implemented according to the following steps:

[0093] (1) First, use the data set A and the factor table A to perform bottom layer module fitting by using the specified "simplified relationship" algorithm B, to see whether the "specified factor relationship" appears (and passes the fitting test of algorithm B). For example, the nonlinear regression fitting of 1 / 2*9.8*X^2 is significant.

[0094] (2) Use the data set A and the factor table A to perform bottom layer module fitting by using the algorithm B, and perform fitting confirmation again, to see whether the "specified factor relationship" appears (and passes the fitting test of algorithm B).

[0095] (3) Then, enter the convolution layer test, first specify the "simplified relationship" of the above specified learning factor, and see whether the "specified factor relationship" appears in the case of only inputting the data set A, to confirm whether the convolution layer and cluster analysis between the algorithm A and the algorithm B are compatible.

[0096] (4) If the specified factor relationship can be calculated in (1)-(3), the correction I is completed. If the specified factor relationship is not calculated, it is confirmed that the fitting module needs to be added, and the fitting module logic is confirmed to be correct. At this time, the algorithm B is used to estimate the minimum amount of data required and adjust the data set A.

[0097] Taking the function 1 / 2gt^2 of the gravity falling distance as an example, the nonlinear regression fitting module of 1 / 2*9.8*X^2 can be added, at this time, the "specified factor relationship" is 1 / 2gX^2, and the "simplified relationship" is the nonlinear regression of the square term of a single factor; or the linear regression fitting module of ln(1 / 2)+ln(9.8)+2ln(t) can be used, at this time, the "specified factor relationship" is ln(1 / 2)+ln(9.8)+2ln(t), and the "simplified relationship" is the ln linear regression of a single factor; or there are g*X relationship fitting modules and ln linear fitting modules, and the combination of similar parameters in the bottom layer is confirmed, at this time, the "specified factor relationship" and the "simplified relationship" change accordingly.

[0098] Take the wind speed moving distance function 1 / 2((1 / 2)C*ρ*S*wind speed*^2 / M)t^2 as an example, a linear regression fitting module of ln(1 / 4)+ln(C)+ln(ρ)+ln(S)+ln(p)+2ln(wind speed)-ln(M)+2ln(t) can be added (as the relative wind speed will be affected as it falls); or a linear fitting module of ln() relationship is confirmed at the bottom.

[0099] Here, after determining that a fitting module needs to be added for algorithm B, the minimum amount of data required can be estimated and the data set A can be adjusted.

[0100] Specifically, after determining the degree of correlation between other factor items and the specified factor item, the statistical analysis amount in each correlation threshold range can be determined according to different degrees of correlation, for example, there is a factor item 1 that is highly correlated with the specified factor item, and there is a factor item 2 that is moderately correlated with the target factor item. Here, a first statistical analysis amount required in a first correlation threshold range indicated by the high correlation can be determined, and a second statistical analysis amount required in a second correlation threshold range indicated by the moderate correlation can also be determined. Then, the statistical analysis amounts are collected to obtain the minimum amount of learning data required to learn the specified factor item. Selecting a first learning data set that matches the minimum amount of learning data from the first learning data set can achieve the adjustment of the first learning data set.

[0101] Second aspect: based on the second algorithm, the quality of the first learning data set as reference model input data is corrected, specifically including the following steps:

[0102] Step one, after determining that the second algorithm is the same as the first algorithm, or after correcting the second algorithm based on the first learning data set, the unlabeled first learning data set is input into the second algorithm for calculation, and the output factor relationship table is obtained;

[0103] Step two, judge whether the output factor relationship table appears the first factor relationship in the first factor relationship table;

[0104] Step three, based on the judgment result, the quality of the first learning data set is adjusted.

[0105] In combination with the above example, the recursive relationship here can be data set A—>algorithm B—>factor table BII, and by comparing with the recursive relationship of data set A—>algorithm B (specified "simplified relationship" algorithm)—>factor table BI, if the factor table BII appears "specified factor relationship" at this time, it can be confirmed that the quality of data set A can be used as a correction template.

[0106] As mentioned above, according to statistical theory and the assumptions behind large models, if the data sets are the same and the algorithms are compatible, the same factor relationship will be calculated in principle.

[0107] After determining the specified factor relationship, the correction of the quality of the data set A can be implemented according to the following steps:

[0108] (1) Data set A—> algorithm B—> factor table BII, to determine whether the specified factor relationship is calculated, and to achieve correction II.

[0109] (2) If correction II is not achieved, the number of data set A is adjusted, and similarly, the minimum number of data required can be evaluated at this time using algorithm B, which will not be described here.

[0110] Third aspect: correcting the second algorithm based on the first learning data set and the second learning data set, specifically including the following steps:

[0111] Step one, mixing the first learning data set and the second learning data set to obtain a mixed learning data set;

[0112] Step two, based on the simplified relationship corresponding to the second algorithm, the second learning data set and the mixed learning data set are calculated to determine the output factor relationship table;

[0113] Step three, determine whether the output factor relationship table appears in the first factor relationship table in the first factor relationship table;

[0114] Step four, based on the judgment result, determine whether the first learning data set and / or the second learning data set needs to be added.

[0115] Among them, the first learning data set and / or the second learning data set in the embodiments of the present application can be added in the following ways:

[0116] Select the learning data to be added from the first learning data set for addition;

[0117] Estimate the missing second learning data based on the first learning data set and the second algorithm;

[0118] Format conversion or other ways for part of the learning data to obtain learning data suitable for mixed analysis of the first learning data set and the second learning data set.

[0119] In combination with the above example, the recursive relationship here can be data set B+data set A—> algorithm B(specified "simplified relationship" algorithm)—> factor table BIII, by comparing with data set A—> algorithm B(specified "simplified relationship" algorithm)—> factor table BI, if the factor table BIII calculates the specified factor relationship, it is confirmed whether the factor table BIII' calculated by data set B—> algorithm B(specified "simplified relationship" algorithm) can also calculate the specified factor relationship.

[0120] If the "specified factor relationship" can be calculated, the confirmation of the partial data missing fitting module or the convolution module in the algorithm B is determined.

[0121] In order to facilitate further understanding of the correction process of the algorithm B, the following steps can be used for specific description:

[0122] (1) Using the data set B + data set A, the algorithm B (specifying the "simplified relationship") is calculated to determine whether the "specified factor relationship" is calculated.

[0123] (2) Using the data set A + data set B and the algorithm A, the factor table A + factor table B is calculated to specify the "simplified relationship" algorithm B containing missing items to perform bottom module fitting, and see whether the "specified factor relationship" appears (and passes the algorithm B fitting test). For example, the partial data is missing wind speed, but the linear regression fitting of ln(1 / 4) + ln(C) + ln(p) + ln(S) + ln(p) + 2ln(wind speed) - ln(M) + 2ln(t) is significant.

[0124] Here, the data set A and the data set B can be different in format or composition, such as the data set A film format A, with wind speed, and the data set B film format B, without wind speed data. At this time, the format conversion and the wind speed data filling are required, but the format conversion can appear to be required to be filled or distorted, and the wind speed data can also be missing. At this time, the algorithm needs to support the algorithm of partial data missing.

[0125] (3) Using the data set A + data set B and the factor table A + factor table B calculated by the algorithm A, the bottom module fitting is performed by the algorithm B, and the fitting is confirmed again to see whether the "specified factor relationship" appears (and passes the algorithm B fitting test).

[0126] (4) Using the mixed data set of data set B + data set A to specify the "simplified relationship" algorithm B of the specified factor, whether the "specified factor relationship" appears, and whether the convolution layer and the cluster analysis between the algorithm A and the algorithm B are compatible.

[0127] (5) Confirming again that only using the data set B -> algorithm B "simplified relationship" confirms the influence of the data set format, and the specified factor relationship can also be calculated, and the correction III is achieved.

[0128] (6) If the correction III is not achieved, it is confirmed that the partial data missing fitting module or the convolution module in the large model algorithm needs to be improved.

[0129] (7) The correction III is achieved, and the simulation filling is used to complete the missing data items.

[0130] The missing data simulation complement in the embodiments of the present application is mainly directed to the data set with the calculated specified factors (non-0), and uses the inverse operation of algorithm B and Bayesian inference to complement the "new data items" related thereto.

[0131] Here, if it is confirmed that the algorithm A and the algorithm B are compatible, the inverse operation of the algorithm A can also be used to fill in when needed. Assuming that the missing data is non-continuous data, the discrete distribution probability is sampled at this time. Or there is non-continuous data in the algorithm, at this time, the Markov chain is used to inverse operate the algorithm B for different discrete data, and the sampling is performed on the operation result according to the distribution probability. If the missing data is only a small number, the missing data is directly fitted, and other related items are the same or are grouped in the specified reliability interval to fill in the missing data by random sampling.

[0132] In specific applications, the missing data in the data set B (the specified factor is not 0) is as much as possible to be supplemented (preferably more than the number of data in the data set A). The missing data here refers to the new data items that are in the data set A but not in the data set B, and the supplement needs to meet the quality standards of the data items in the data set A. If there is no suitable data source, the data item can be blank. For example, in the following example, the gravity and wind speed of the self-measured image measured land are measured, or the data of the weather bureau at the time and place are supplemented.

[0133] The data set A is shown in the following table:

[0134] The data set B after the new data items is shown in the following table:

[0135] The fourth aspect: based on the second algorithm and the first learning data set, the quality of the second learning data set is corrected, and the specific steps include the following steps:

[0136] Step one, mixing the first learning data set and the second learning data set to obtain a mixed learning data set;

[0137] Step two, inputting the mixed learning data set into the second algorithm to perform operation and determining the output factor relationship table;

[0138] Step three, judging whether the output factor relationship table appears in the first factor relationship table in the first factor relationship table;

[0139] Step four, adjusting the quality of the second learning data set based on the judgment result.

[0140] Among them, the adjustment of the quality of the second learning data set can be specifically realized according to the following steps:

[0141] Step one, if the first factor relationship in the first factor relationship table does not appear, the second algorithm is used to label the mixed learning data set composed of the second learning data set and the first learning data set;

[0142] Step two, remove the second learning data whose specified factor label does not conform to the specified factor label distribution of the first learning data set from the second learning data set, and obtain the updated mixed learning data set;

[0143] Step three, input the updated mixed learning data set into the second algorithm again for calculation, and determine whether the output factor relationship table appears the first factor relationship in the first factor relationship table;

[0144] Step four, if the first factor relationship in the first factor relationship table still does not appear, the corresponding second learning data is excluded by random sampling or stratified sampling to obtain the updated mixed learning data set with emerging new relationships or uniformly irrelevant relationships; wherein the excluded data set is retained for later large model iterative update.

[0145] In combination with the above example, the recursive relationship here can be data set B + data set A—> algorithm B—> factor table BIV. By comparing with the recursive relationship of data set A—> algorithm B—> factor table BII, if factor table BIV calculates the “specified factor relationship” at this time, the quality of data set B is confirmed. This mainly considers that the large model will group the data, and according to the statistical theorem and the assumption behind the large model, if data set A and the algorithm are compatible, the specified factor relationship can be calculated, and the mixed data set of data set A and data set B and the algorithm are compatible. In principle, the same factor relationship will also be calculated. This correction method can correct the model drift that may be generated by the data set.

[0146] In order to further understand the correction process of the quality of data set B, the following steps can be used to specifically explain:

[0147] (1) data set B + data set A—> algorithm B—> factor table BIV. If the specified factor item is calculated, the correction IV is completed at this time.

[0148] If the correction IV is not completed, the learning data that does not meet the quality standard of data set A needs to be excluded and the data distribution deviation needs to be adjusted. Data set B is fitted with data set A—> algorithm B one by one, and factor table BIV is used as an index for adjustment:

[0149] (2) If the correction IV is not completed, the single item data of data set B (the specified factor is not 0) is calculated, and the related factor exceeds the specified confidence interval. This single item data is directly removed from data set B and included in the re-inspection data set at this time. At this time, data set can also be supplemented by extracting data from the standby data library.

[0150] The statistical algorithm here will have a bias to estimate the factor value of the specified factor. For example, the factor relationship table BII has (X + x variation) ^ 2 + (Y + y variation) + the rest of the variation. Assuming X is the specified factor, Y is the related factor, and Y is assumed to be the missing item, the Y' value is inversely calculated by algorithm B, and then the y variation and the assumed distribution of the model cluster can be calculated to calculate the confidence interval (Y'-a*y variation < Y < Y'+b*y variation).

[0151] If it is discrete data, it is the possible value range of certain data. At this time, the factors other than the specified probability are excluded (such as 1%), and the probability is lower than the probability by drawing lots, and then the related factors of continuous data are excluded.

[0152] If the correction IV is still not completed, compare factor table BIV with factor table BII, and use random sampling or stratified sampling to exclude the distribution of the specified factor, so that the overall data set is properly distributed. Here, the excluded data is included in the standby data set.

[0153] If the correction IV is still not completed, compare factor table BIV with factor table BII, and use random sampling or stratified sampling to exclude the distribution of the specified factor, so that the overall data set is properly distributed. Here, the excluded data is included in the standby data set.

[0154] (5) If the above exclusion still cannot pass the correction, there may be other relationships between factors, and further Bayesian algorithm can be used to exclude other distribution outliers (2) based on the parameters calculated by algorithm B from data set A. The data set is included in the test data set.

[0155] The Bayesian algorithm here refers to calculating the probability of the data set appearing based on the prior probability of the distribution of the specified factor and the related factor. For example, based on the prior probability obtained from factor table BII, A is the specified factor, C and D are the related factors, the probability of the C and D values appearing is calculated, and the probability outliers are excluded.

[0156] The above (2)-(5) can be automatically excluded, and after exclusion, enter the standby data library or the test data library. Manual inspection or other detection is further confirmed whether to be included again (need to compare correction IV and correction II, can output comprehensive image or factor distribution, manual judgment is assisted).

[0157] Among them, the order of (2)-(5) can be changed, and at the same time, if it is confirmed that algorithm A and algorithm B are compatible, algorithm A can also be used to replace algorithm B for automatic exclusion when needed.

[0158] It can be seen that the large model correction method provided by the embodiment of the application adopts a specified factor relationship as an index, and through the above four correction operations, the large model A is used to automatically confirm the to-be-corrected points and automatically adjust the data set of the large model B, so as to achieve the correction mode.

[0159] Here, the specified factor relationship is verified (the specified factor relationship can be verified through a linear regression or a fitting module or a convolution module, etc.), so as to be flexibly applicable to cross-model correction. The template large model A can be artificially specified or collected, so that the model B can be corrected in a scientific manner, in accordance with reality, or in accordance with other artificial target requirements.

[0160] It should be emphasized that the embodiment of the application can be applied to different situations such as correction of different models by the same reference large model, correction of a single model by different reference large models, or correction of different model agents by different reference large models, formation of agents large model, and the like. It is also applicable to different stages of large models, external supervision or external consultant assistance (through external provision of templates and internal generation of specified factor relationship proofs). In addition, the correctable items, targets, and principles of each step are determined, but the order can be adjusted for different use scenarios. At the same time, an automatic algorithm program test and data set supplement or removal are provided.

[0161] In summary, the large model correction method provided by the embodiment of the application mainly has the following technical advantages:

[0162] (1) Current model correction is mainly performed in the early stage (such as before going online), but existing large models still produce model drift and some errors with the feeding of data. The embodiment of the application provides a systematic method for confirmation, provides different adjustment methods and standards for different correction stages, can correct the large model back or partially correct it back to the preset reference model, can be used for correction at each stage of model development, and avoids model drift and the like.

[0163] (2) Existing large models mainly use artificial learning data filtering or semi-automatic data selection standards, but the embodiment of the application can specify factor correction, which can be used in various different situations, and can be automatically or semi-automatically used in specific implementation. The use method is flexible. At the same time, not only data items but also algorithms are corrected, so that inverse deduction or rule change correction of the model becomes possible.

[0164] (3) If the embodiment of the application is a large model correction that specifies scientific principles as specified factors, the model can be close to reality. If the large model corrected according to the scientific function meets the scientific principles, various technical applications can be further fitted, and more realistic and feasible applications can be selected to avoid being divorced from reality.

[0165] (4) The embodiment of the application corrects the large model based on the specified factor, can timely correct the application rules of the large model, so that the large model can respond more quickly to environmental and rule changes, and is more adaptable.

[0166] In the description of the present specification, the description referring to the terms "some possible embodiments", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application, and the above terms do not necessarily represent the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any suitable manner in any one or more embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the specification and the features of the different embodiments or examples without contradiction.

[0167] Regarding the method flowchart of the embodiments of the present application, some operations are described as different steps executed in a certain order. Such flowcharts are illustrative rather than limiting. Some steps described herein can be grouped together and executed in a single operation, or some steps can be divided into multiple sub-steps, and some steps can be executed in an order different from that shown herein. Each step shown in the flowchart can be implemented in any way by any circuit structure and / or tangible mechanism (for example, by software running on a computer device, hardware (for example, processor or chip implemented logic function), etc., and / or any combination thereof) in any manner.

[0168] Those skilled in the art can understand that in the method described in the above specific embodiments, the writing order of each step does not mean strict execution order, and the specific execution order of each step should be determined by its function and possible inherent logic.

[0169] Based on the same inventive concept, the embodiments of the present application also provide a large model correction device corresponding to the large model correction method. Since the principle of the device in the embodiments of the present application solves the problem similar to the above-mentioned large model correction method of the embodiments of the present application, the implementation of the device can be referred to the implementation of the method, and the repeated parts will not be described.

[0170] Referring to FIG. 2, it is a schematic diagram of a large model correction device provided by the embodiments of the present application. The device comprises an acquisition module 201, a correction module 202, a determination module 203; wherein,

[0171] The acquisition module 201 is configured to acquire reference large model data and / or to-be-corrected large model data; the reference large model data comprises a first learning data set and a first algorithm, and the to-be-corrected large model data comprises a second learning data set and a second algorithm.

[0172] The correction module 202 is configured to perform a correction operation on the to-be-corrected large model according to at least one or more of the following correction manners:

[0173] correcting the second algorithm based on the first learning data set, correcting the quality of the first learning data set as reference model input data based on the second algorithm, correcting the second algorithm based on the first learning data set and the second learning data set, and correcting the quality of the second learning data set based on the second algorithm and the first learning data set.

[0174] The determination module 203 is configured to obtain a corrected large model based on the at least one or more correction operations.

[0175] The above large model correction device can perform a correction operation on the to-be-corrected large model according to at least one or more of the following correction manners: the correction operation includes but is not limited to correcting the second algorithm based on the first learning data set, correcting the quality of the first learning data set as reference model input data based on the second algorithm, correcting the second algorithm based on the first learning data set and the second learning data set, and correcting the quality of the second learning data set based on the second algorithm and the first learning data set. It can be seen that, based on the comparison of the reference large model, the corrected large model obtained by correction is greatly enhanced in the learning data set and the algorithm, which provides good model support for subsequent specific applications downstream, and is more practical.

[0176] Optionally, the reference large model data further comprises a first factor relationship table, and the first factor relationship table is used to represent the association relationship between each first factor; the to-be-corrected large model data further comprises a second factor relationship table, and the second factor relationship table is used to represent the association relationship between each second factor.

[0177] Optionally, the correction module 202 is specifically configured to correct the second algorithm based on the first learning data set according to the following steps:

[0178] performing label annotation on the first learning data set by using the first algorithm to obtain a labeled first learning data set;

[0179] performing calculation on the labeled first learning data set based on a simplified relationship corresponding to the second algorithm to determine an output factor relationship table, and determining whether the output factor relationship table appears in the first factor relationship table.

[0180] determining whether the output factor relationship table includes the first factor relationship in the first factor relationship table;

[0181] determining whether a fitting module needs to be added for the second algorithm based on the determination result, the fitting module being used for fitting the first factor relationship.

[0182] Optionally, the correction module 202 is further configured to, after determining that the fitting module needs to be added for the second algorithm, determine a minimum amount of data required for fitting the first factor relationship based on the fitting module, and adjust the first learning data set based on the minimum amount of data to obtain an adjusted first learning data set.

[0183] Optionally, the correction module 202 is specifically configured to correct the quality of the first learning data set as reference model input data of the second algorithm based on the following steps:

[0184] after determining that the second algorithm is the same as the first algorithm or after correcting the second algorithm based on the first learning data set, inputting the unlabeled first learning data set into the second algorithm to perform calculation, and obtaining an output factor relationship table;

[0185] determining whether the output factor relationship table includes the first factor relationship in the first factor relationship table;

[0186] adjusting the quality of the first learning data set based on the determination result.

[0187] Optionally, the correction module 202 is specifically configured to correct the second algorithm based on the first learning data set and the second learning data set according to the following steps:

[0188] mixing the first learning data set and the second learning data set to obtain a mixed learning data set;

[0189] performing calculation on the second learning data set and the mixed learning data set based on a simplified relationship corresponding to the second algorithm to determine an output factor relationship table;

[0190] determining whether the output factor relationship table includes the first factor relationship in the first factor relationship table;

[0191] determining whether the first learning data set and / or the second learning data set needs to be added based on the determination result.

[0192] Optionally, the correction module 202 is specifically configured to add the first learning data set and / or the second learning data set according to at least one or more of the following ways:

[0193] selecting learning data to be added from the first learning data set;

[0194] estimating missing second learning data based on the first learning data set and the second algorithm;

[0195] performing format conversion or other operations on part of the learning data to obtain learning data suitable for mixed analysis of the first learning data set and the second learning data set.

[0196] Optionally, in the case where the learning data to be added is non-continuous data, the correction module 202 is specifically configured to select the learning data to be added from the first learning data set in one of the following manners:

[0197] sampling based on discrete distribution probability;

[0198] performing inverse operation on the second algorithm based on Markov chain and sampling the operation result based on distribution probability.

[0199] Optionally, the correction module 202 is specifically configured to correct the quality of the second learning data set in the following steps:

[0200] mixing the first learning data set and the second learning data set to obtain a mixed learning data set;

[0201] inputting the mixed learning data set into the second algorithm to perform operation and determine an output factor relationship table;

[0202] judging whether the output factor relationship table contains the first factor relationship in the first factor relationship table;

[0203] adjusting the quality of the second learning data set based on the judgment result.

[0204] Optionally, the correction module 202 is specifically configured to adjust the quality of the second learning data set based on the judgment result in the following steps:

[0205] if the first factor relationship in the first factor relationship table does not appear, performing label annotation on the mixed learning data set composed of the second learning data set and the first learning data set by using the second algorithm;

[0206] removing the second learning data with a specified factor label that does not conform to the specified factor label distribution of the first learning data set from the second learning data set, and obtaining an updated mixed learning data set;

[0207] inputting the updated mixed learning data set into the second algorithm to perform operation again, and determining whether the output factor relationship table contains the first factor relationship in the first factor relationship table;

[0208] If the first factor relationship in the first factor relationship table still does not appear, the corresponding second learning data is screened out in a random sampling manner or a stratified sampling manner to obtain an updated mixed post-learning data set with emerging new relationships or uniformly irrelevant relationships; wherein, the screened-out data set is reserved for later large model iterative updating.

[0209] It should be noted that the device in the embodiments of the present application can implement each process of the embodiments of the foregoing method and achieve the same effects and functions, which will not be described here.

[0210] The embodiments of the present application also provide an electronic device, as shown in FIG. 3, which is a structural schematic diagram of an electronic device provided by the embodiments of the present application, including a processor 301, a memory 302, and a bus 303. The memory 302 stores machine-readable instructions executable by the processor 301 (such as the execution instructions corresponding to the acquisition module 201, the correction module 202, and the determination module 203 in the device in FIG. 2, etc.), when the electronic device is running, the processor 301 and the memory 302 communicate through the bus 303, and the machine-readable instructions are executed by the processor 301 to perform the following processing:

[0211] Obtain reference large model data and / or to-be-corrected large model data; the reference large model data includes a first learning data set and a first algorithm, and the to-be-corrected large model data includes a second learning data set and a second algorithm;

[0212] Perform the correction operation of the to-be-corrected large model in at least one or more of the following correction modes:

[0213] Correct the second algorithm based on the first learning data set, correct the quality of the first learning data set as reference model input data based on the second algorithm, correct the second algorithm based on the first learning data set and the second learning data set, and correct the quality of the second learning data set based on the second algorithm and the first learning data set;

[0214] Based on at least one or more correction operations, obtain a corrected large model.

[0215] The embodiments of the present application also provide a computer-readable storage medium, which stores a computer program, and the computer program is run by a processor to execute the steps of the large model correction method described in the method embodiment. Wherein, the storage medium can be a volatile or non-volatile computer-readable storage medium.

[0216] The embodiment of the present application further provides a computer program product carrying program codes, the program codes include instructions for executing steps of the large model correction method described in the above method embodiments, and details can be referred to the above method embodiments, which will not be repeated here.

[0217] The computer program product can be implemented by hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium. In another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK) or the like.

[0218] The embodiments in the present application are described in a progressive manner, and the same or similar parts of each embodiment can be referred to each other. Each embodiment mainly describes the difference from other embodiments. Especially, the device, equipment and computer readable storage medium embodiments are basically similar to the method embodiments, so the description is simplified, and the related parts can be referred to the description of the method embodiments.

[0219] The device, equipment and computer readable storage medium provided by the embodiments of the present application correspond to the method, so the device, equipment and computer readable storage medium also have the similar beneficial technical effects as the method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the device, equipment and computer readable storage medium will not be repeated here.

[0220] Those skilled in the art should understand that the embodiments of the present application can be implemented as a method and device (equipment or system), or a computer readable storage medium. Therefore, the present application can be implemented as a complete hardware embodiment, a complete software embodiment, or a combination of software and hardware. Moreover, the present application can be implemented in the form of a computer readable storage medium implemented on one or more computer readable storage media (including but not limited to disk memory, compact disk memory (CD-ROM), optical memory, etc.) containing computer usable program codes.

[0221] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks.

[0222] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks.

[0223] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks.

[0224] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0225] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) and / or cache memory, for storing instructions and data used by one or more of the components of the computing device. The memory can further include non-volatile memory, such as read-only memory (ROM), electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or non-volatile random access memory (NVRAM) for storing instructions and data used by one or more of the components of the computing device. For example, the memory can include a hard disk or other magnetic, optical, electromagnetic, solid state, or other device used for storage.

[0226] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer-readable storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory, read-only memory, electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device. In addition, although the operations of the method of the present application are described in a specific order in the accompanying drawings, this does not require or imply that the operations must be performed in that specific order, or that all shown operations must be performed to achieve the desired results. In addition, some steps can be omitted, a plurality of steps can be combined into one step, and / or one step can be divided into a plurality of sub-steps.

[0227] Although the spirit and principles of the present application have been described above with reference to several specific embodiments, it should be understood that the present application is not limited to the disclosed specific embodiments, and the division of aspects does not mean that the features in these aspects cannot be combined. The present application is intended to cover various modifications and equivalent arrangements included in the spirit and scope of the appended claims.

Claims

1. A large model calibration method comprising: Obtaining reference large model data and / or large model data to be corrected; the reference large model data includes a first learning data set and a first algorithm, and the large model data to be corrected includes a second learning data set and a second algorithm; Perform correction operations on the large model to be corrected according to at least one or more of the following correction methods: Calibrate the second algorithm based on the first learning data set, calibrate the quality of the first learning data set as reference model input data based on the second algorithm, calibrate the second algorithm based on the first learning data set and the second learning data set, and calibrate the quality of the second learning data set based on the second algorithm and the first learning data set; Based on at least one or more correction operations, a corrected large model is obtained.

2. The method according to claim 1, wherein The reference large model data also includes a first factor relationship table, which is used to characterize the association relationship between each first factor; the large model data to be corrected also includes a second factor relationship table, which is used to characterize the association relationship between each second factor.

3. The method according to claim 2, wherein: The calibrating the second algorithm based on the first learning data set includes: labeling the first learning data set using the first algorithm to obtain a labeled first learning data set; Calculating the labeled first learning data set based on the simplified relationship corresponding to the second algorithm to determine an output factor relationship table; and determining whether the output factor relationship table contains the first factor relationship in the first factor relationship table; Calculating the labeled first learning data set based on the second algorithm to determine an output factor relationship table; and determining whether the output factor relationship table contains the first factor relationship in the first factor relationship table; Based on the determination result, it is determined whether a fitting module needs to be added to the second algorithm, where the fitting module is used to fit the first factor relationship.

4. The method according to claim 3, wherein: After determining that a fitting module needs to be added to the second algorithm, the method further includes: Determining the minimum amount of data required to fit the first factor relationship based on the fitting module; The first learning material set is adjusted based on the minimum amount of materials to obtain an adjusted first learning material set.

5. The method according to any one of claims 2 to 4, wherein Correcting the quality of the first learning data set as reference model input data based on the second algorithm includes: After determining that the second algorithm is the same as the first algorithm, or after calibrating the second algorithm based on the first learning data set, inputting the unlabeled first learning data set into the second algorithm for calculation to obtain an output factor relationship table; Determine whether the output factor relationship table contains the first factor relationship in the first factor relationship table; The quality of the first learning material set is adjusted based on the judgment result.

6. The method according to any one of claims 2 to 5, wherein The calibrating the second algorithm based on the first learning data set and the second learning data set includes: Mixing the first learning data set and the second learning data set to obtain a mixed learning data set; Calculating the second learning data set and the mixed learning data set based on the simplified relationship corresponding to the second algorithm to determine an output factor relationship table; Determine whether the output factor relationship table contains the first factor relationship in the first factor relationship table; Based on the judgment result, it is determined whether the first learning material set and / or the second learning material set needs to be supplemented.

7. The method according to claim 6, wherein: The first learning material set and / or the second learning material set are supplemented in at least one or more of the following ways: Selecting learning materials that need to be added from the first learning material set and adding them; estimating missing second learning data based on the first learning data set and a second algorithm; Part of the learning materials is formatted or otherwise converted to obtain learning materials suitable for mixed analysis of the first learning material set and the second learning material set.

8. The method according to claim 6 or 7, wherein: In the case that the learning materials to be added are non-continuous materials, the learning materials to be added are selected from the first learning material set in one of the following ways: Sampling based on discrete distribution probabilities; An inverse operation is performed on the second algorithm based on a Markov chain, and the operation result is sampled with a distribution probability.

9. The method according to any one of claims 2 to 8, wherein Correcting the quality of the second learning data set includes: Mixing the first learning data set and the second learning data set to obtain a mixed learning data set; Inputting the mixed learning data set into the second algorithm for calculation to determine an output factor relationship table; Determine whether the output factor relationship table contains the first factor relationship in the first factor relationship table; The quality of the second learning material set is adjusted based on the judgment result.

10. The method according to claim 9, wherein: The adjusting the quality of the second learning material set based on the judgment result includes: If the first factor relationship in the first factor relationship table does not appear, labeling the mixed learning data set composed of the second learning data set and the first learning data set using a second algorithm; removing the second learning data whose designated factor labels do not conform to the designated factor label distribution of the first learning data set from the second learning data set, and obtaining an updated mixed learning data set; Inputting the updated mixed learning data set into the second algorithm again for calculation, and determining whether the output factor relationship table contains the first factor relationship in the first factor relationship table; If the first factor relationship in the first factor relationship table still does not appear, the corresponding second learning data are screened out by random sampling or stratified sampling to obtain a mixed learning data set with new relationships or non-correlated relationships evenly updated; among them, the screened and excluded data set is retained for subsequent iterative updates of the large model.

Images