Model blood relationship determination method and device, program product and equipment

Abstract

The invention provides a model blood relationship determination method and device, a computer program product and electronic equipment, and relates to the technical field of artificial intelligence. The model blood relationship determination method comprises the following steps: acquiring a plurality of evaluation data sets; respectively inputting the plurality of evaluation data sets into the plurality of evaluation models to obtain model evaluation results of each evaluation model under the plurality of evaluation data sets; determining the similarity of the evaluation results of each evaluation model under the plurality of evaluation data sets according to the model evaluation results of each evaluation model under the plurality of evaluation data sets; according to the evaluation result similarity of each evaluation model under the plurality of evaluation data sets, the blood relationship among the evaluation models is determined, and the convenience and reliability of model blood relationship analysis can be improved to a certain extent.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a method for determining a model blood relationship, a device for determining a model blood relationship, a computer program product, and an electronic device. Background Art

[0002] With the development of artificial intelligence technology, various general-purpose large models and industry large models have emerged one after another. There is no clear identification of the relationship between different large models and software systems, making it difficult to determine whether they are self-developed or secondary development based on open source frameworks. As a result, it is difficult to determine the transparency and credibility of large models when conducting large model evaluations.

[0003] In related technologies, when determining the blood relationship of models, the sensitive data features of the large model are extracted in an engineering and systematic manner through observation comparison of the generated stress data to form a test load, and the test load is used to carry out security testing on the large model to verify whether there is a blood relationship between different large models. This method requires a separate feature analysis for each large model, which is relatively complicated.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The present disclosure provides a model blood relationship determination method, a model blood relationship determination device, a computer program product and an electronic device, so as to at least to some extent solve the problem that model blood relationship determination in related technologies is relatively complicated.

[0006] According to a first aspect of the present disclosure, a method for determining model blood relationship is provided, the method comprising: acquiring multiple evaluation data sets; inputting the multiple evaluation data sets into multiple evaluation models respectively to obtain model evaluation results of each evaluation model under the multiple evaluation data sets; determining the similarity of the evaluation results of each evaluation model under the multiple evaluation data sets based on the model evaluation results of each evaluation model under the multiple evaluation data sets; determining the blood relationship between each evaluation model based on the similarity of the evaluation results of each evaluation model under the multiple evaluation data sets.

[0007] According to a second aspect of the present disclosure, a device for determining a model blood relationship is provided, the device comprising: an evaluation result determination module, used to obtain multiple evaluation data sets; input the multiple evaluation data sets into multiple evaluation models respectively to obtain model evaluation results of each evaluation model under the multiple evaluation data sets; a similarity determination module, used to determine the similarity of the evaluation results of each evaluation model under the multiple evaluation data sets based on the model evaluation results of each evaluation model under the multiple evaluation data sets; and a blood relationship determination module, used to determine the blood relationship between each evaluation model based on the similarity of the evaluation results of each evaluation model under the multiple evaluation data sets.

[0008] According to a third aspect of the present disclosure, a computer program product is provided, including a computer program, which, when executed by a processor, implements the model blood relationship determination method of the above-mentioned first aspect and its possible implementation methods.

[0009] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the model blood relationship determination method of the above-mentioned first aspect and its possible implementation method by executing the executable instructions.

[0010] The technical solution disclosed in this disclosure has the following beneficial effects:

[0011] In the process of determining the above-mentioned model blood relationship, multiple evaluation data sets are obtained; the multiple evaluation data sets are respectively input into multiple evaluation models to obtain model evaluation results of each evaluation model under multiple evaluation data sets; based on the model evaluation results of each evaluation model under multiple evaluation data sets, the similarity of the evaluation results of each evaluation model under multiple evaluation data sets is determined; based on the similarity of the evaluation results of each evaluation model under multiple evaluation data sets, the blood relationship between each evaluation model is determined. The present disclosure analyzes the blood relationship of the model based on the evaluation results of different evaluation models under multiple evaluation data sets, which can not only improve the convenience and reliability of model blood relationship analysis, but also assist in the selection of model bases, ensure that the development team protects and verifies the technical originality and intellectual property compliance of its own large models, and ensures independent control. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 A flow chart showing a method for determining a model blood relationship in this exemplary embodiment;

[0013] Figure 2 A schematic diagram of constructing an evaluation result vector in this exemplary embodiment is shown;

[0014] Figure 3A schematic diagram showing a comparison of evaluation result change trends of different evaluation models in various subdivision dimensions in this exemplary embodiment;

[0015] Figure 4 A flowchart showing a model blood relationship determination in this exemplary embodiment is shown;

[0016] Figure 5 A system architecture diagram showing a model blood relationship determination in this exemplary embodiment;

[0017] Figure 6 A structural block diagram of a model blood relationship determination device in this exemplary embodiment is shown;

[0018] Figure 7 An electronic device for implementing the above-mentioned model blood relationship determination method in this exemplary embodiment is shown. DETAILED DESCRIPTION

[0019] Exemplary embodiments of the present disclosure will be described more fully hereinafter with reference to the accompanying drawings.

[0020] The accompanying drawings are schematic diagrams of the present disclosure and are not necessarily drawn to scale. Some of the block diagrams shown in the accompanying drawings may be functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in hardware modules or integrated circuits, or in networks, processors or microcontrollers. The embodiments can be implemented in various forms and should not be construed as being limited to the examples set forth herein. The features, structures or characteristics described in the present disclosure may be combined in one or more embodiments in any suitable manner. In the description below, many specific details are provided to provide a full description of the embodiments of the present disclosure. However, those skilled in the art should appreciate that one or more specific details may be omitted when implementing the technical solution of the present disclosure, or one or more specific details may be replaced by other methods, components, devices, steps, etc.

[0021] In related technologies, when determining the blood relationship of models, the sensitive data features of the large model are extracted in an engineering and systematic manner through observation comparison of the generated stress data to form a test load, and the test load is used to carry out security testing on the large model to verify whether there is a blood relationship between different large models. This method requires a separate feature analysis for each large model, which is relatively complicated.

[0022] In view of the above problems, exemplary embodiments of the present disclosure provide a model blood relationship determination method, a model blood relationship determination processing device, a computer program product, and an electronic device.

[0023] In an optional embodiment, reference Figure 1 A method for determining a model blood relationship is provided, which specifically includes the following steps S110 to S130:

[0024] Step S110, obtaining multiple evaluation data sets; inputting the multiple evaluation data sets into multiple evaluation models respectively, and obtaining model evaluation results of each evaluation model under the multiple evaluation data sets;

[0025] Step S120, determining the similarity of the evaluation results of each evaluation model under the multiple evaluation data sets according to the model evaluation results of each evaluation model under the multiple evaluation data sets;

[0026] Step S130, determining the blood relationship between the evaluation models according to the similarity of the evaluation results of each evaluation model under multiple evaluation data sets.

[0027] Figure 1 In the method shown, the model's lineage relationship is analyzed based on the evaluation results of different evaluation models under multiple evaluation data sets. This can not only improve the convenience and reliability of model lineage analysis, but also assist in the selection of model bases, ensuring that the development team protects and verifies the technical originality and intellectual property compliance of its own large models, and ensures independent control.

[0028] Below Figure 1 Provide detailed instructions for each step.

[0029] In step S110, a plurality of evaluation data sets are obtained; the plurality of evaluation data sets are respectively input into a plurality of evaluation models to obtain model evaluation results of each evaluation model under the plurality of evaluation data sets.

[0030] The evaluation data set refers to the data set used to evaluate the model, which can be constructed in advance. For example, the evaluation data set can be constructed based on the benchmark data released by domestic and foreign scientific research institutions and / or leading enterprises, so that the evaluation results are objective, and the model kinship determination based on this is also objective.

[0031] Optionally, the multiple evaluation data sets may include general evaluation data sets and industry evaluation data sets. Among them, the general evaluation data set refers to an evaluation data set that is applicable to various industry backgrounds and is not restricted by industry. Among them, the industry evaluation data set refers to an evaluation data set constructed for a specific industry background. According to the industry background corresponding to the required evaluation model, an industry evaluation data set matching the industry can be selected.

[0032] Optionally, multiple general evaluation data sets and / or multiple industry evaluation data sets can be used to evaluate the evaluation model. The data sets cover general industries and vertical industries, so that the evaluation results are comprehensive, and the model kinship determination based on this is also comprehensive. It should be noted that the kinship, i.e., "homology" in this disclosure, refers to the similarity between the large model and the software system in terms of model architecture, code structure, parameter weights, etc., which can be used to determine whether the large model is self-developed or secondary developed based on open source frameworks.

[0033] The evaluation model can be a large model or software system that needs to be evaluated. It should be noted that large models usually have machine learning models with ultra-large-scale parameters (for example, more than one billion) and complex computing structures, which can process massive amounts of data and complete various complex tasks, such as natural language processing and image recognition.

[0034] Specifically, after obtaining multiple evaluation data sets, the multiple evaluation data sets can be input into multiple evaluation models for operation to obtain the model evaluation results of each evaluation model under the multiple evaluation data sets. By inputting the evaluation data sets into the evaluation models and analyzing the evaluation results, the evaluation method is simple and convenient.

[0035] Exemplarily, evaluation data set 1, evaluation data set 2, ..., evaluation data set n can be respectively input into evaluation model A to obtain model evaluation results A1, model evaluation results A2, ..., model evaluation results An of evaluation model A under evaluation data set 1, evaluation data set 2, ..., evaluation data set n; evaluation data set 1, evaluation data set 2, ..., evaluation data set n can be respectively input into evaluation model B to obtain model evaluation results B1, model evaluation results B2, ..., model evaluation results Bn of evaluation model B under evaluation data set 1, evaluation data set 2, ..., evaluation data set n; in this way, the model evaluation results of each evaluation model under evaluation data set 1, evaluation data set 2, ..., evaluation data set n can be obtained.

[0036] Optionally, after each evaluation data set is run in the evaluation model, the scoring result of each subdivision dimension can be obtained based on the degree of matching between the evaluation model calculation results and the standard results, such as standardization, friendliness, regularity, etc.

[0037] Optionally, the large model can be evaluated from different dimensions according to the built-in algorithm corresponding to each evaluation data set, and the final calculated evaluation score can be given to obtain the model evaluation result.

[0038] In an optional implementation, when the evaluation items corresponding to the evaluation data set include multiple evaluation dimensions, the multiple evaluation data sets are respectively input into multiple evaluation models to obtain model evaluation results of each evaluation model under the multiple evaluation data sets. This can be achieved by the following steps: the multiple evaluation data sets are respectively input into multiple evaluation models to obtain model evaluation results of each evaluation model under each evaluation dimension corresponding to the multiple evaluation data sets.

[0039] Optionally, the evaluation item corresponding to the evaluation data set may include multiple evaluation dimensions, and each evaluation dimension may be further divided into multiple subdivision dimensions. The number of subdivision dimensions divided by different evaluation dimensions may be consistent or inconsistent, and this disclosure does not specifically limit this.

[0040] For example, the evaluation items of evaluation data set 1 have m evaluation dimensions, wherein the first evaluation dimension has s subdivision dimensions, and the mth evaluation dimension has t subdivision dimensions, where m, s, and t are all integers. Figure 2 As shown, the evaluation data set 1 can be input into the evaluation model A to obtain the model evaluation results A11, A12, ..., A1s, ..., Am1, Am2, ..., Amt of the evaluation model A in each evaluation dimension. Among them, Amt represents the model evaluation result of the evaluation model A in the tth subdivision dimension of the mth evaluation dimension.

[0041] Evaluating the evaluation model from multiple evaluation dimensions can improve the detail and accuracy of the evaluation results.

[0042] In step S120, the similarity of the evaluation results of each evaluation model under the multiple evaluation data sets is determined according to the model evaluation results of each evaluation model under the multiple evaluation data sets.

[0043] Exemplarily, under evaluation data set 1, the model evaluation results corresponding to evaluation model A, evaluation model B, ..., evaluation model K are A1, B1, ..., K1 respectively. The model evaluation results corresponding to different evaluation models can be compared in pairs to obtain the similarity between A1 and B1, the similarity between A1 and K1, and so on, and so on, to obtain the similarity of the evaluation results of each evaluation model under multiple evaluation data sets.

[0044] In an optional implementation, the above-mentioned determination of the similarity of the evaluation results of each evaluation model under multiple evaluation data sets based on the model evaluation results of each evaluation model under multiple evaluation data sets can be achieved through the following steps: constructing a vector group for each evaluation model according to the model evaluation results of each evaluation model under a target evaluation data set to obtain a vector group of each evaluation model under the target evaluation data set; the target evaluation data is any data set among the multiple evaluation data sets; performing similarity judgment on the vector group of each evaluation model under the target evaluation data set to determine the vector similarity between each evaluation model under the target evaluation data set; determining the similarity of the evaluation results between each evaluation model under the target evaluation data set according to the vector similarity between each evaluation model under the target evaluation data set.

[0045] Specifically, the model evaluation results of each evaluation model under each evaluation dimension in the target evaluation data set can be used to generate a vector group. Figure 2 As shown, according to the model evaluation results A11, A12, ..., A1s, ..., Am1, Am2, ..., Amt of the evaluation model A under each evaluation dimension corresponding to the evaluation data set 1, a vector group {A11, A12, ...A1s, ...Am1, Am2, ...Amt} can be constructed for the evaluation model A. Among them, Amt represents the model evaluation result of the evaluation model A under the t-th subdivision dimension of the m-th evaluation dimension corresponding to the evaluation data set 1.

[0046] In an optional implementation, the above-mentioned similarity judgment of the vector groups of each evaluation model under the target evaluation data set and determination of the vector similarity between each evaluation model under the target evaluation data set can be achieved through the following steps: determining the vector change trend of each evaluation model under the target evaluation data set based on the vector group of each evaluation model under the target evaluation data set; determining the vector similarity between each evaluation model under the target evaluation data set by comparing the vector change trend of each evaluation model under the target evaluation data set.

[0047] For example, the change trends of the evaluation result vectors of different evaluation models under each evaluation dimension or subdivision dimension can be compared to determine the vector similarity. Figure 3 The figure provides a schematic diagram comparing the changing trends of evaluation results of different evaluation models in each segment dimension, among which the changing trends of evaluation results of evaluation model A and evaluation model B in each segment dimension are similar, and the vector similarity between evaluation model A and evaluation model B is higher; the changing trends of evaluation results of evaluation model C, evaluation model A and evaluation model B in each segment dimension are different, and the vector similarity between evaluation model C, evaluation model A and evaluation model B is lower.

[0048] Exemplarily, methods such as cosine similarity may also be used to calculate the vector similarity between evaluation models under the target evaluation data set.

[0049] If two evaluation models are related, even if the evaluation results change due to adjustments to parameters, the change trends of their evaluation results will still be similar and the vector similarity will still be high.

[0050] After determining the vector similarity between each evaluation model under the target evaluation data set, the vector similarity between each evaluation model under the target evaluation data set can be used to characterize the similarity of the evaluation results between each evaluation model under the target evaluation data set. This process does not require a separate feature analysis for each evaluation model, which can improve the convenience of blood relationship determination to a certain extent.

[0051] In step S130, the blood relationship between the evaluation models is determined according to the similarity of the evaluation results of each evaluation model under multiple evaluation data sets.

[0052] Specifically, the similarity of the evaluation results of each evaluation model under multiple evaluation data sets can be used to determine whether there is a blood relationship, that is, "homology", between the evaluation models. The higher the similarity of the evaluation results, the greater the possibility of a blood relationship.

[0053] In an optional implementation, the above-mentioned determination of the blood relationship between the evaluation models based on the similarity of the evaluation results of each evaluation model under multiple evaluation data sets can be achieved through the following steps: determining the overall similarity between the evaluation models based on the similarity of the evaluation results of each evaluation model under multiple evaluation data sets; determining the blood relationship between the evaluation models based on the overall similarity between the evaluation models.

[0054] Exemplarily, the similarities of evaluation results between evaluation model A and evaluation model B in evaluation data set 1, evaluation data set 2, ..., evaluation data set n are S1, S2, ..., Sn respectively. The overall similarity between evaluation model A and evaluation model B can be obtained by taking the average of S1, S2, ..., Sn.

[0055] Exemplarily, the model evaluation results of evaluation model A under evaluation data set 1, evaluation data set 2, ..., evaluation data set n are A1, A2, ..., An respectively, and the model evaluation results of evaluation model B under evaluation data set 1, evaluation data set 2, ..., evaluation data set n are B1, B2, ..., Bn respectively. It can be judged whether A1 and B1, A2 and B2, ..., An and Bn are similar, and the percentage of the number of similar groups to the total number of comparison groups n is taken as the overall similarity between evaluation model A and evaluation model B.

[0056] Based on the similarity of the evaluation results of each evaluation model under multiple evaluation data sets, the overall similarity between the evaluation models is determined, which can improve the reliability of model similarity analysis to a certain extent.

[0057] It should be noted that when the similarity of evaluation results between evaluation models under multiple evaluation data sets is represented by the vector similarity between evaluation models under multiple evaluation data sets, the overall similarity between evaluation models may be the overall vector similarity between evaluation models.

[0058] In an optional implementation, the above-mentioned determination of the blood relationship between the evaluation models based on the overall similarity between the evaluation models can be achieved through the following steps: judging whether the overall similarity between the first evaluation model and the second evaluation model exceeds a preset threshold; wherein the first evaluation model and the second evaluation model are two models among multiple evaluation models; if the overall similarity between the first evaluation model and the second evaluation model exceeds the preset threshold, determining that there is a blood relationship between the first evaluation model and the second evaluation model; if the overall similarity between the first evaluation model and the second evaluation model does not exceed the preset threshold, determining that there is no blood relationship between the first evaluation model and the second evaluation model.

[0059] The preset threshold may be a preset similarity threshold. Specifically, if the overall similarity between the two evaluation models exceeds the preset threshold, it can be considered that the two evaluation models are related by blood; if the overall similarity between the two evaluation models does not exceed the preset threshold, it can be considered that the two evaluation models are not related by blood.

[0060] Exemplarily, the model evaluation results of evaluation model A in evaluation data set 1, evaluation data set 2, ..., evaluation data set n are A1, A2, ..., An, respectively, and the model evaluation results of evaluation model B in evaluation data set 1, evaluation data set 2, ..., evaluation data set n are B1, B2, ..., Bn, respectively. It is judged whether A1 and B1, A2 and B2, ..., An and Bn are similar. If the number of similar groups exceeds the preset number of groups, or the percentage of similar groups in the total comparison group number n exceeds the preset percentage, it can be considered that evaluation model A and evaluation model B are similar as a whole and have a blood relationship.

[0061] By comparing with the preset threshold, it is helpful to determine the model lineage and judge whether the model is self-developed or secondary development based on an open source framework.

[0062] like Figure 4 The figure provides a flow chart for determining the blood relationship of a model, which may specifically include the following steps:

[0063] Step S401, obtaining multiple evaluation data sets;

[0064] Step S402, by looping through these evaluation data sets, inputting these evaluation data sets into multiple evaluation models respectively, and obtaining model evaluation results of each evaluation model under each evaluation dimension corresponding to these evaluation data sets;

[0065] Step S403, generating a vector group from the model evaluation results of each evaluation model under each evaluation dimension corresponding to these evaluation data sets, and obtaining the vector group of each evaluation model under these evaluation data sets;

[0066] Step S404, performing similarity judgment on the vector groups of each evaluation model under these evaluation data sets, and determining the vector similarity between each evaluation model under these evaluation data sets;

[0067] Step S405, determining the overall vector similarity between the evaluation models according to the vector similarity between the evaluation models under the evaluation data sets;

[0068] Step S406, determining whether the overall vector similarity between the evaluation models exceeds a preset threshold; if so, executing step S407; if not, executing step S408;

[0069] Step S407, there is a blood relationship;

[0070] Step S408: There is no blood relationship.

[0071] Figure 4 In the steps shown, the model lineage relationships are analyzed based on the evaluation results of different evaluation models under multiple evaluation data sets. This can not only improve the convenience and reliability of model lineage analysis, but also assist in the selection of model bases, ensuring that the development team protects and verifies the technical originality and intellectual property compliance of its own large models, and ensures independent control.

[0072] like Figure 5As shown, a system architecture diagram for determining model blood relationship is provided, which may include a vector group generation module, a vector similarity calculation module, and a blood relationship determination module. In the actual application process, the evaluation data set (which may include a general data set and an industry data set) may be input into the evaluation model to be determined in advance for operation, and the evaluation results may be output; the vector group generation module may generate evaluation result vector groups for the evaluation results of each evaluation model under the evaluation data set; the vector similarity calculation module may perform vector similarity (such as cosine similarity, etc.) calculation on the vector groups of different evaluation models under the same evaluation data set, record the vector similarity value, and obtain the evaluation result vector group of each evaluation model under each evaluation data set by looping through all evaluation data sets, and perform vector similarity calculation one by one; the blood relationship determination module may calculate the overall vector similarity based on the vector similarity of any two evaluation models under all evaluation data sets, and determine whether it exceeds the threshold value. If it exceeds the threshold value, it may be determined that the two evaluation models have a blood relationship, and if it does not exceed the threshold value, it may be determined that the two evaluation models do not have a blood relationship.

[0073] The exemplary embodiment of the present disclosure also provides a model blood relationship determination device. Figure 6 As shown, the model blood relationship determination device 600 may include the following program modules:

[0074] The evaluation result determination module 610 is used to obtain multiple evaluation data sets; input the multiple evaluation data sets into multiple evaluation models respectively, and obtain the model evaluation results of each evaluation model under the multiple evaluation data sets;

[0075] A similarity determination module 620 is used to determine the similarity of the evaluation results of each evaluation model under multiple evaluation data sets according to the model evaluation results of each evaluation model under multiple evaluation data sets;

[0076] The blood relationship determination module 630 is used to determine the blood relationship between the evaluation models according to the similarity of the evaluation results of each evaluation model under multiple evaluation data sets.

[0077] In an optional implementation, based on the above solution, the multiple evaluation data sets include a general evaluation data set and an industry evaluation data set.

[0078] In an optional implementation, based on the aforementioned scheme, the evaluation items corresponding to the evaluation data set include multiple evaluation dimensions, and the evaluation result determination module 610 can be configured to: input the multiple evaluation data sets into multiple evaluation models respectively, and obtain the model evaluation results of each evaluation model under each evaluation dimension corresponding to the multiple evaluation data sets.

[0079] In an optional embodiment, based on the aforementioned scheme, the similarity determination module 620 includes: a vector group construction module, which is used to construct a vector group for each evaluation model according to the model evaluation results of each evaluation model under the target evaluation data set, and obtain the vector group of each evaluation model under the target evaluation data set; the target evaluation data is any data set among multiple evaluation data sets; a vector group similarity judgment module, which is used to perform similarity judgment on the vector groups of each evaluation model under the target evaluation data set, and determine the vector similarity between each evaluation model under the target evaluation data set; an evaluation result similarity judgment module, which is used to determine the similarity of evaluation results between each evaluation model under the target evaluation data set according to the vector similarity between each evaluation model under the target evaluation data set.

[0080] In an optional implementation, based on the aforementioned scheme, the vector group similarity judgment module can be configured to: determine the vector change trend of each evaluation model under the target evaluation data set according to the vector group of each evaluation model under the target evaluation data set; determine the vector similarity between each evaluation model under the target evaluation data set by comparing the vector change trend of each evaluation model under the target evaluation data set.

[0081] In an optional embodiment, based on the aforementioned scheme, the blood relationship determination module 630 includes: an overall similarity determination module, which is used to determine the overall similarity between each evaluation model according to the similarity of the evaluation results of each evaluation model under multiple evaluation data sets; and a blood relationship recognition module, which is used to determine the blood relationship between each evaluation model according to the overall similarity between each evaluation model.

[0082] In an optional embodiment, based on the aforementioned scheme, the blood relationship identification module can be configured to: determine whether the overall similarity between the first evaluation model and the second evaluation model exceeds a preset threshold; wherein the first evaluation model and the second evaluation model are two models among multiple evaluation models; if the overall similarity between the first evaluation model and the second evaluation model exceeds the preset threshold, it is determined that there is a blood relationship between the first evaluation model and the second evaluation model; if the overall similarity between the first evaluation model and the second evaluation model does not exceed the preset threshold, it is determined that there is no blood relationship between the first evaluation model and the second evaluation model.

[0083] The specific details of each part of the above-mentioned model blood relationship determination device 600 have been described in detail in the implementation method part. The undisclosed details can be found in the implementation method part, so they will not be repeated here.

[0084] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the exemplary embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0085] The exemplary embodiments of the present disclosure also provide a computer program product, which includes a computer program, and when the computer program is executed by a processor, the above-mentioned model blood relationship determination method is implemented.

[0086] In one embodiment, the computer program product may be a tangible product containing a computer program, such as a computer-readable storage medium storing a computer program. The readable storage medium may be a storage medium based on electrical, magnetic, optical, electromagnetic, infrared, or other signals, including but not limited to: random access memory (RAM), read-only memory (ROM), magnetic tape, floppy disk, flash memory (Flash), mechanical hard disk (HDD), solid-state drive (SSD), and the like. Exemplarily, the computer program product may be implemented as a non-volatile storage medium storing a computer program, such as a read-only memory, a NAND flash memory, and the like.

[0087] In one embodiment, the computer program product may be an intangible product including a computer program. Exemplarily, the computer program product may be implemented as a virtual digital product, such as a digital file storing an executable file, an installation package, etc. of the computer program.

[0088] The code of the computer program can be written in one or more programming languages. Programming languages ​​such as C language, Java, C++, etc. The program code can be executed entirely on the user computing device, or partially on the user computing device, or as a separate software package, or partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, such as a local area network (LAN), a wide area network (WAN), etc., or can be connected to an external computing device (e.g., an Internet connection provided by an operator).

[0089] The computer program may be carried or transmitted via electrical, magnetic, optical, electromagnetic, infrared, or other signals. The electronic device may convert the signal carrying the computer program into a digital signal, thereby running the computer program. When the computer program is run on an electronic device, its code is used to enable the electronic device to execute (more specifically, the processor of the electronic device may execute) the method steps of various exemplary embodiments of the present disclosure, such as the above method, which may include the following steps:

[0090] Acquire multiple evaluation data sets; input the multiple evaluation data sets into multiple evaluation models respectively to obtain model evaluation results of each evaluation model under the multiple evaluation data sets;

[0091] Determine the similarity of the evaluation results of each evaluation model under the multiple evaluation data sets according to the model evaluation results of each evaluation model under the multiple evaluation data sets;

[0092] According to the similarity of the evaluation results of each evaluation model under multiple evaluation data sets, the blood relationship between the evaluation models is determined.

[0093] In an optional implementation, based on the above solution, the multiple evaluation data sets include a general evaluation data set and an industry evaluation data set.

[0094] In an optional implementation, based on the aforementioned scheme, the evaluation items corresponding to the evaluation data set include multiple evaluation dimensions, and the multiple evaluation data sets are respectively input into multiple evaluation models to obtain model evaluation results of each evaluation model under the multiple evaluation data sets. This can be achieved by the following steps: multiple evaluation data sets are respectively input into multiple evaluation models to obtain model evaluation results of each evaluation model under each evaluation dimension corresponding to the multiple evaluation data sets.

[0095] In an optional implementation, based on the aforementioned scheme, according to the model evaluation results of each evaluation model under multiple evaluation data sets, determining the similarity of the evaluation results of each evaluation model under multiple evaluation data sets can be achieved through the following steps: according to the model evaluation results of each evaluation model under the target evaluation data set, constructing a vector group for each evaluation model to obtain the vector group of each evaluation model under the target evaluation data set; the target evaluation data is any one of the multiple evaluation data sets; performing similarity judgment on the vector groups of each evaluation model under the target evaluation data set to determine the vector similarity between each evaluation model under the target evaluation data set; according to the vector similarity between each evaluation model under the target evaluation data set, determining the similarity of the evaluation results between each evaluation model under the target evaluation data set.

[0096] In an optional implementation, based on the aforementioned scheme, the vector groups of each evaluation model under the target evaluation data set are judged for similarity, and the vector similarities between the evaluation models under the target evaluation data set are determined. This can be achieved by the following steps: based on the vector groups of each evaluation model under the target evaluation data set, the vector change trends of each evaluation model under the target evaluation data set are determined; by comparing the vector change trends of each evaluation model under the target evaluation data set, the vector similarities between the evaluation models under the target evaluation data set are determined.

[0097] In an optional implementation, based on the aforementioned scheme, the blood relationship between the evaluation models is determined according to the similarity of the evaluation results of each evaluation model under multiple evaluation data sets, which can be achieved through the following steps: determine the overall similarity between the evaluation models according to the similarity of the evaluation results of each evaluation model under multiple evaluation data sets; determine the blood relationship between the evaluation models according to the overall similarity between the evaluation models.

[0098] In an optional implementation, based on the aforementioned scheme, the blood relationship between the evaluation models is determined according to the overall similarity between the evaluation models, which can be achieved through the following steps: judging whether the overall similarity between the first evaluation model and the second evaluation model exceeds a preset threshold; wherein the first evaluation model and the second evaluation model are two models among multiple evaluation models; if the overall similarity between the first evaluation model and the second evaluation model exceeds the preset threshold, determining that there is a blood relationship between the first evaluation model and the second evaluation model; if the overall similarity between the first evaluation model and the second evaluation model does not exceed the preset threshold, determining that there is no blood relationship between the first evaluation model and the second evaluation model.

[0099] In the process of determining the above-mentioned model lineage relationships, the model lineage relationships are analyzed based on the evaluation results of different evaluation models under multiple evaluation data sets. This can not only improve the convenience and reliability of model lineage analysis, but also assist in the selection of model bases, ensuring that the development team protects and verifies the technical originality and intellectual property compliance of its own large models, and ensures independent control.

[0100] The exemplary embodiment of the present disclosure also provides an electronic device capable of implementing the above-mentioned model blood relationship determination method. The electronic device may include a processor and a memory. The memory stores executable instructions of the processor, such as program code. The processor executes the method in this exemplary embodiment by executing the executable instructions. In addition, the electronic device may also include a display for displaying a graphical user interface.

[0101] Reference below Figure 7 , the electronic device is exemplarily described in the form of a general-purpose computing device. It should be understood that Figure 7The electronic device 700 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0102] like Figure 7 As shown, the electronic device 700 may include: a processor 710 , a memory 720 , a bus 730 , an I / O (input / output) interface 740 , a network adapter 750 , and a display 760 .

[0103] The memory 720 may include a volatile memory, such as a RAM 721, a cache unit 722, and may also include a non-volatile memory, such as a ROM 723. The memory 720 may also include one or more program modules 724, such program modules 724 include but are not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or a combination thereof may include the implementation of a network environment. For example, the program module 724 may include each module in the above-mentioned device.

[0104] The processor 710 may include one or more processing units. For example, the processor 710 may include an AP (Application Processor), a modem processor, a GPU (Graphics Processing Unit), an ISP (Image Signal Processor), a controller, an encoder, a decoder, a DSP (Digital Signal Processor), a baseband processor and / or an NPU (Neural-Network Processing Unit), etc.

[0105] The processor 710 may be configured to execute executable instructions stored in the memory 720 , such as executing any one or more method steps in the exemplary embodiment.

[0106] Exemplarily, the processor 710 may perform the following steps:

[0107] Acquire multiple evaluation data sets; input the multiple evaluation data sets into multiple evaluation models respectively to obtain model evaluation results of each evaluation model under the multiple evaluation data sets;

[0108] Determine the similarity of the evaluation results of each evaluation model under the multiple evaluation data sets according to the model evaluation results of each evaluation model under the multiple evaluation data sets;

[0109] According to the similarity of the evaluation results of each evaluation model under multiple evaluation data sets, the blood relationship between the evaluation models is determined.

[0110] In an optional implementation, based on the above solution, the multiple evaluation data sets include a general evaluation data set and an industry evaluation data set.

[0111] In an optional implementation, based on the aforementioned scheme, the evaluation items corresponding to the evaluation data set include multiple evaluation dimensions, and the multiple evaluation data sets are respectively input into multiple evaluation models to obtain model evaluation results of each evaluation model under the multiple evaluation data sets. This can be achieved by the following steps: multiple evaluation data sets are respectively input into multiple evaluation models to obtain model evaluation results of each evaluation model under each evaluation dimension corresponding to the multiple evaluation data sets.

[0112] In an optional implementation, based on the aforementioned scheme, according to the model evaluation results of each evaluation model under multiple evaluation data sets, determining the similarity of the evaluation results of each evaluation model under multiple evaluation data sets can be achieved through the following steps: according to the model evaluation results of each evaluation model under the target evaluation data set, constructing a vector group for each evaluation model to obtain the vector group of each evaluation model under the target evaluation data set; the target evaluation data is any one of the multiple evaluation data sets; performing similarity judgment on the vector groups of each evaluation model under the target evaluation data set to determine the vector similarity between each evaluation model under the target evaluation data set; according to the vector similarity between each evaluation model under the target evaluation data set, determining the similarity of the evaluation results between each evaluation model under the target evaluation data set.

[0113] In an optional implementation, based on the aforementioned scheme, the vector groups of each evaluation model under the target evaluation data set are judged for similarity, and the vector similarities between the evaluation models under the target evaluation data set are determined. This can be achieved by the following steps: based on the vector groups of each evaluation model under the target evaluation data set, the vector change trends of each evaluation model under the target evaluation data set are determined; by comparing the vector change trends of each evaluation model under the target evaluation data set, the vector similarities between the evaluation models under the target evaluation data set are determined.

[0114] In an optional implementation, based on the aforementioned scheme, the blood relationship between the evaluation models is determined according to the similarity of the evaluation results of each evaluation model under multiple evaluation data sets, which can be achieved through the following steps: determine the overall similarity between the evaluation models according to the similarity of the evaluation results of each evaluation model under multiple evaluation data sets; determine the blood relationship between the evaluation models according to the overall similarity between the evaluation models.

[0115] In an optional implementation, based on the aforementioned scheme, the blood relationship between the evaluation models is determined according to the overall similarity between the evaluation models, which can be achieved through the following steps: judging whether the overall similarity between the first evaluation model and the second evaluation model exceeds a preset threshold; wherein the first evaluation model and the second evaluation model are two models among multiple evaluation models; if the overall similarity between the first evaluation model and the second evaluation model exceeds the preset threshold, determining that there is a blood relationship between the first evaluation model and the second evaluation model; if the overall similarity between the first evaluation model and the second evaluation model does not exceed the preset threshold, determining that there is no blood relationship between the first evaluation model and the second evaluation model.

[0116] In the process of determining the above-mentioned model lineage relationships, the model lineage relationships are analyzed based on the evaluation results of different evaluation models under multiple evaluation data sets. This can not only improve the convenience and reliability of model lineage analysis, but also assist in the selection of model bases, ensuring that the development team protects and verifies the technical originality and intellectual property compliance of its own large models, and ensures independent control.

[0117] The bus 730 is used to realize the connection between different components of the electronic device 700, and may include a data bus, an address bus, and a control bus.

[0118] The electronic device 700 can communicate with one or more external devices 800 (eg, a keyboard, a mouse, an external controller, etc.) through the I / O interface 740 .

[0119] The electronic device 700 can communicate with one or more networks through the network adapter 750. For example, the network adapter 750 can provide mobile communication solutions such as 3G / 4G / 5G, or wireless communication solutions such as wireless LAN, Bluetooth, near field communication, etc. The network adapter 750 can communicate with other modules of the electronic device 700 through the bus 730.

[0120] The electronic device 700 may display a graphical user interface and the like through the display 760 .

[0121] although Figure 7 Not shown, other hardware and / or software modules may also be provided in the electronic device 700, including but not limited to: a display, a microcode, a device driver, a redundant processor, an external disk drive array, a RAID (Redundant Arrays of Independent Disks) system, a tape drive, and a data backup storage system.

[0122] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the exemplary embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0123] It will be appreciated by those skilled in the art that various aspects of the present disclosure may be implemented as a system, method or program product. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which may be collectively referred to herein as a "circuit", "module" or "system". Those skilled in the art will readily think of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any variations, uses or adaptive changes of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and implementation are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0124] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for determining a model blood relationship, characterized in that: The method comprises: Acquire multiple evaluation data sets; input the multiple evaluation data sets into multiple evaluation models respectively to obtain model evaluation results of each evaluation model under the multiple evaluation data sets; Determine the similarity of the evaluation results of each evaluation model under the multiple evaluation data sets according to the model evaluation results of each evaluation model under the multiple evaluation data sets; The blood relationship between the evaluation models is determined according to the similarity of the evaluation results of the evaluation models under the multiple evaluation data sets.

2. The method according to claim 1, characterized in that The multiple evaluation data sets include general evaluation data sets and industry evaluation data sets.

3. The method according to claim 1, characterized in that The evaluation items corresponding to the evaluation data set include multiple evaluation dimensions, and the multiple evaluation data sets are respectively input into multiple evaluation models to obtain model evaluation results of each evaluation model under the multiple evaluation data sets, including: The multiple evaluation data sets are respectively input into multiple evaluation models to obtain model evaluation results of each evaluation model under each evaluation dimension corresponding to the multiple evaluation data sets.

4. The method according to claim 1, characterized in that: Determining the similarity of the evaluation results of each evaluation model under the multiple evaluation data sets according to the model evaluation results of each evaluation model under the multiple evaluation data sets includes: According to the model evaluation results of each evaluation model under the target evaluation data set, construct a vector group for each evaluation model to obtain a vector group of each evaluation model under the target evaluation data set; the target evaluation data is any data set among the multiple evaluation data sets; Performing similarity judgment on the vector groups of each evaluation model under the target evaluation data set, and determining the vector similarity between each evaluation model under the target evaluation data set; According to the vector similarities between the evaluation models under the target evaluation data set, the similarities of the evaluation results between the evaluation models under the target evaluation data set are determined.

5. The method according to claim 4, characterized in that The step of performing similarity judgment on the vector groups of each evaluation model under the target evaluation data set to determine the vector similarity between each evaluation model under the target evaluation data set includes: Determine, according to the vector groups of each evaluation model under the target evaluation data set, the vector change trend of each evaluation model under the target evaluation data set; By comparing the vector change trends of each evaluation model under the target evaluation data set, the vector similarity between each evaluation model under the target evaluation data set is determined.

6. The method according to claim 1, characterized in that Determining the blood relationship between the evaluation models according to the similarity of the evaluation results of the evaluation models under the multiple evaluation data sets includes: Determining the overall similarity between the evaluation models according to the similarity of the evaluation results of the evaluation models under the multiple evaluation data sets; The blood relationship between the evaluation models is determined according to the overall similarity between the evaluation models.

7. The method according to claim 6, characterized in that Determining the blood relationship between the evaluation models according to the overall similarity between the evaluation models includes: Determine whether the overall similarity between the first evaluation model and the second evaluation model exceeds a preset threshold; wherein the first evaluation model and the second evaluation model are two models among the multiple evaluation models; If the overall similarity between the first evaluation model and the second evaluation model exceeds a preset threshold, it is determined that there is a blood relationship between the first evaluation model and the second evaluation model; If the overall similarity between the first evaluation model and the second evaluation model does not exceed a preset threshold, it is determined that there is no blood relationship between the first evaluation model and the second evaluation model.

8. A model blood relationship determination device, characterized in that: The device comprises: An evaluation result determination module is used to obtain multiple evaluation data sets; input the multiple evaluation data sets into multiple evaluation models respectively, and obtain model evaluation results of each evaluation model under the multiple evaluation data sets; A similarity determination module, used to determine the similarity of the evaluation results of each evaluation model under the multiple evaluation data sets according to the model evaluation results of each evaluation model under the multiple evaluation data sets; The blood relationship determination module is used to determine the blood relationship between the evaluation models according to the similarity of the evaluation results of each evaluation model under the multiple evaluation data sets.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 7 by executing the executable instructions.

Images