Large model blood relationship analysis method and system

Abstract

The invention discloses a large model blood relationship analysis method and system, and belongs to the field of artificial intelligence, and the method comprises the steps: processing a weight matrix of a target AI model, and extracting the fingerprint features of the model; further analyzing the characteristics of the weight matrix of the AI model; extracting the hierarchical structure, the layer type and the number of neurons in each layer of the model to obtain architecture information of the model; comparing the fingerprint information between the two AI models with the feature information of the models, and judging the similarity of the architecture; further analyzing whether the models have a blood relationship or not by comparing weight matrixes of corresponding layers of the two AI models; a similarity threshold is set according to the weight matrix analysis structure of the corresponding layer, and whether the blood relationship exists or not is judged through the similarity threshold. Through an automatic blood relationship tracking mode, the blood relationship analysis efficiency and accuracy are remarkably improved. An automatic tool is used for tracking data, parameters and calculation flow directions among all layers in the model, manual intervention is reduced, and the accuracy and the high efficiency of analysis are ensured.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and more specifically, particularly relates to a large model bloodline analysis method and system. Background Art

[0002] With the rapid development of artificial intelligence (AI) and deep learning technology, modern AI models are becoming increasingly large and complex, especially in the fields of natural language processing, computer vision, speech recognition, etc. The application of large-scale models such as deep neural networks (DNN) and generative adversarial networks (GAN) has reached an unprecedented level. In order to improve the performance of the model and optimize the utilization of computing resources, developers need to perform detailed analysis, optimization and management of these large models. Model Lineage Analysis, as an important technical means, is mainly used to trace and analyze the source, structure, weight and similarity of AI models and their components.

[0003] Model lineage analysis not only helps developers understand the structural changes of the model, but also optimizes the migration, reuse and verification process between models. However, there is a lack of technical patents or research on large-scale model lineage analysis. Especially in the process of large-scale model training, reasoning and iterative optimization, there is almost no systematic solution for comprehensive tracking and analysis of lineage relationships. With the continuous development of large models, especially in the collaborative work of multiple sub-models, across stages and across services, how to effectively track and analyze the lineage relationships of models has become a difficult problem that needs to be solved urgently. Therefore, a large model lineage analysis method is proposed, which can not only make up for the shortcomings of existing technologies, but also provide strong support for the optimization and management of large-scale deep learning models. Summary of the invention

[0004] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0005] In view of the problems of the above-mentioned or existing large-model bloodline analysis methods and systems, the present invention is proposed.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: The embodiment of the present invention provides a method for analyzing the lineage of a large model, including: processing a weight matrix of a target AI model to extract fingerprint features of the model; Further analyze the characteristics of the weight matrix of the AI ​​model; Extract the model's hierarchical structure, layer type, and number of neurons in each layer to obtain the model's architecture information; By comparing the fingerprint information and feature information of two AI models, the similarity of their architectures can be determined; We then further analyze whether the models are related by comparing the weight matrices of the corresponding layers of the two AI models; The similarity threshold is set according to the weight matrix analysis structure of the corresponding layer, and whether there is a blood relationship is determined by the similarity threshold.

[0007] As a preferred solution of the large model lineage analysis method of the present invention, the weight matrix of the target AI model is processed to extract the fingerprint features of the model, including: Assuming that the weight matrix of one layer of the model is W, its corresponding fingerprint Flayer can be calculated as follows: Among them, W i is the weight element in the weight matrix, n is the total number of elements in the weight matrix, and sigmoid is an activation function used to normalize and map the weights; Concatenate the features of all layers to convert the weight information of the entire model into a high-dimensional vector. If the model has multiple layers such as convolutional layers and fully connected layers, extract fingerprints from the weight matrix of each layer separately, and then concatenate the fingerprint features of each layer into a long vector to form a high-dimensional feature vector Fmodel, which is used to uniquely represent the model: Among them, L represents the number of layers of the large model, F i Represents the fingerprint of each layer of the large model.

[0008] As a preferred solution of the large model blood relationship analysis method of the present invention, the weight matrix of the AI ​​model is further analyzed to obtain its characteristics, including: Use the clustering algorithm to analyze the weight matrix, divide the eigenvectors of the weight matrix into K clusters, find out the feature differences between different clusters, and then identify the features related to the model behavior. The basic formula of the K-means algorithm is: Suppose there is a set of feature data (x1,x2,...,xn), where each data point , and select K cluster centers (c1,c2,...,cK), and divide them by minimizing the sum of the squared distances between each data point and its cluster center: Where I(ci=k) is the indicator function, its value is 1 when (ci=k), otherwise it is 0; In each iteration, the mean of each cluster is calculated and the data points are redistributed to the nearest cluster center until convergence. By performing K-means clustering analysis on the weight matrix, the central features of each cluster are obtained, and then the features related to the model behavior are identified.

[0009] As a preferred solution of the large model lineage analysis method of the present invention, wherein: the hierarchical structure, layer type and number of neurons in each layer of the model are extracted to obtain the architecture information of the model, including: Calculate the difference between the weight matrix before and after fine-tuning, using the L2 norm distance measurement method; perform cluster analysis on the fine-tuned weight matrix to identify the newly introduced features during the fine-tuning process; mark the fine-tuned layer and compare it with the unfine-tuned layer to analyze its impact on the overall lineage of the model.

[0010] As a preferred solution of the large model lineage analysis method of the present invention, wherein: comparing the fingerprint information and the feature information of the two AI models to determine the similarity of their architectures includes: Compare the total number of layers L1 and L2 of the two models. If L1=L2, the architectures are considered similar. Compare the types of each layer, such as convolutional layer, fully connected layer, and pooling layer. If the types of corresponding layers are consistent, the architectures are further considered to be similar; For each layer, the number of neurons in the two models is compared. If the number of neurons in the two models is the same, the layer is considered similar; The similarity of the model architecture is quantified by defining a similarity metric function Sarch: Where L is the total number of layers of the two models, n 1,l and n 2,l are the number of neurons in the lth layer of model 1 and model 2 respectively. The closer the Sarch value is to 1, the more similar the two model architectures are. The weight similarity of the two models is further analyzed.

[0011] As a preferred solution of the large model blood relationship analysis method of the present invention, the weight matrices of the corresponding layers of the two AI models are compared to further analyze whether the models are blood related: Compare the weight similarity of the two models layer by layer. For the weight matrices W1 and W2 of the corresponding layers of the two models, calculate the corresponding fingerprint feature vectors F1 and F2 through model fingerprint extraction, and achieve weight similarity matching by calculating the similarity between the two fingerprint feature vectors; The similarity is calculated using the cosine similarity method, which evaluates the similarity by calculating the cosine value of the angle between two vectors. The calculation formula is as follows: Among them, F 1i , F 2i Represent the components of F1 and F2 respectively. The value range of cosine similarity is between -1 and 1. When the cosine similarity is close to 1, it means that the two vectors are highly similar; when the cosine similarity is close to -1, it means that there is no similarity between the two vectors.

[0012] The similarity of the weight matrices corresponding to each layer of the two models is calculated in turn, and then the average is calculated to obtain the similarity of the two models for subsequent blood relationship analysis.

[0013] As a preferred solution of the large model blood relationship analysis method of the present invention, wherein: the similarity threshold is set according to the weight matrix analysis structure of the corresponding layer, and whether there is a blood relationship is determined by the similarity threshold, including: According to the similarity calculation results, a similarity threshold is set. The similarity between the two models is quantified by the similarity threshold, and it is judged whether they are related by blood. If the similarity is greater than the threshold, the models are considered to be related by blood. The similarity threshold A is set in the following range: A>0.9 indicates high similarity: it means that the models are highly similar in structure, parameters or data; A between 0.7 and 0.9 indicates moderate similarity: it means that there is a certain degree of similarity between the models; A>0.5 indicates low similarity: it means that there is basic similarity between the models; Suppose we analyze two neural network models A and B to determine whether they are related. The similarity between the two models is calculated to be 0.85: If the similarity threshold is set to 0.8, then since 0.85>0.8, it is concluded that models A and B are related by blood; If the similarity is lower than the set threshold, for example, 0.65, then since 0.65<0.8, it is concluded that models A and B are not related by blood.

[0014] An analysis system based on large model lineage includes: a fingerprint feature extraction module for processing the weight matrix of the target AI model and extracting the fingerprint features of the model; Feature analysis module, used to further analyze the characteristics of the weight matrix of the AI ​​model; The model architecture acquisition module is used to extract the model's hierarchical structure, layer type, and number of neurons in each layer to obtain the model's architecture information; The architecture judgment module is used to judge the similarity of the architectures of two AI models by comparing the fingerprint information and feature information of the models; The weight matrix judgment module further analyzes whether the models are related by comparing the weight matrices of the corresponding layers of the two AI models; The threshold determination module is used to set a similarity threshold according to the weight matrix analysis structure of the corresponding layer, and determine whether there is a blood relationship through the similarity threshold.

[0015] A computing device, comprising: at least one processor, memory, and input-output unit; The memory is used to store computer programs, and the processor is used to call the computer programs stored in the memory to execute the steps of the large model bloodline analysis method.

[0016] A computer-readable storage medium includes instructions, which, when executed on a computer, cause the computer to perform the steps of a large model lineage analysis method.

[0017] The beneficial effects of the present invention are as follows: the present invention significantly improves the efficiency and accuracy of bloodline analysis through automated bloodline tracking. The present invention uses automated tools to track the data, parameters and calculation flows between various levels in the model in real time, reducing manual intervention and ensuring the accuracy and efficiency of the analysis. Through technical features such as automated bloodline analysis and model fingerprint extraction, many pain points of the existing technology in large-scale model development are solved, work efficiency, model interpretability, computing resource optimization and team collaboration capabilities are improved, and significant technological advancement is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0019] Figure 1 A flowchart of a large model bloodline analysis method provided in an embodiment of the present invention.

[0020] Figure 2 A schematic diagram of the structure of a large-model bloodline analysis system provided in an embodiment of the present invention.

[0021] Figure 3 A schematic structural diagram of a medium according to an embodiment of the present invention is schematically shown.

[0022] Figure 4 A schematic diagram of the structure of a computing device according to an embodiment of the present invention is schematically shown.

[0023] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION

[0024] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0025] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0026] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0027] Example Reference below Figure 1 , Figure 1 This is a flow chart of a large model lineage analysis method provided by an embodiment of the present invention. It should be noted that the embodiments of the present invention can be applied to any applicable scenario.

[0028] Figure 1 The process of the large model blood relationship analysis method provided by one embodiment of the present invention shown includes: S1: Process the weight matrix of the target AI model to extract the fingerprint features of the model.

[0029] Preferably, assuming that the weight matrix of one layer of the model is W, its corresponding fingerprint Flayer can be calculated as follows: Among them, W i is the weight element in the weight matrix, n is the total number of elements in the weight matrix, and sigmoid is an activation function used to normalize and map the weights; Concatenate the features of all layers to convert the weight information of the entire model into a high-dimensional vector. If the model has multiple layers such as convolutional layers and fully connected layers, extract fingerprints from the weight matrix of each layer separately, and then concatenate the fingerprint features of each layer into a long vector to form a high-dimensional feature vector Fmodel, which is used to uniquely represent the model: Among them, L represents the number of layers of the large model, F iRepresents the fingerprint of each layer of the large model.

[0030] Furthermore, by concatenating the fingerprint feature vectors F1, F2, …, FL of each layer, a high-dimensional vector Fmodel is obtained, which uniquely identifies the structure and weight of the model. This fingerprint vector can distinguish between different models and provide a unique identifier for the model.

[0031] Assume that there are multiple models that have been trained or fine-tuned differently. The relationship between them is determined by comparing the similarity between the fingerprint vectors Fmodel of these models. If the fingerprint vectors of two models are very similar, then their structures and weights may also be very similar. They may belong to the same type of model, or one is a fine-tuned version of the other. By comparing the fingerprint features of the model before and after fine-tuning, it can be determined whether the fine-tuning operation has significantly changed the structure or parameters of the model. If the fine-tuning operation has little effect on the fingerprint vector, it means that the fine-tuning mainly adjusts certain weights, and the overall structure of the model has not changed significantly. If the fingerprint after fine-tuning is significantly different from the fingerprint of the original model, it can be inferred that the fine-tuning has significantly changed the structure of the model.

[0032] S2: Further analyze the characteristics of the weight matrix of the AI ​​model.

[0033] Preferably, a clustering algorithm is used to analyze the weight matrix, divide the eigenvectors of the weight matrix into K clusters, find out the feature differences between different clusters, and then identify the features related to the model behavior. The basic formula of the K-means algorithm is: Suppose there is a set of feature data (x1,x2,...,xn), where each data point , and select K cluster centers (c1,c2,...,cK), and divide them by minimizing the sum of the squared distances between each data point and its cluster center: Where I(ci=k) is the indicator function, its value is 1 when (ci=k), otherwise it is 0; In each iteration, the mean of each cluster is calculated and the data points are redistributed to the nearest cluster center until convergence. By performing K-means clustering analysis on the weight matrix, the central features of each cluster are obtained, and then the features related to the model behavior are identified.

[0034] Further, assume that there is a deep neural network model whose weight matrices are W1, W2, …, WL, where the weight matrix Wl of each layer is a matrix of ml×nl, representing the weight of the layer. In this embodiment, the weight matrix of each layer is regarded as a feature vector and expanded into a long vector W. For the weight matrix Wl of each layer, it is expanded into a one-dimensional vector Wl′, that is, each element in the matrix is ​​spliced ​​into a vector in rows or columns: For each layer of the network, calculate its expanded weight vector Wl′, and concatenate the weight vectors of all layers into a data set X={W1′,W2′,…,WL′}. Each row of this data set is the weight feature vector of a layer.

[0035] Suppose we choose to divide the eigenvectors of the weight matrix into K clusters. We randomly initialize K cluster centers C={c1,c2,…,cK}, where each cluster center ck is a vector of the same dimension as the eigenvector of the weight matrix.

[0036] For each data point xi, assign it to the nearest cluster according to the distance to each cluster center ck. That is, select the cluster center that minimizes the distance: For each cluster k, update the cluster center ck to the mean of all data points in the cluster: Among them, S k is the set of all data points in cluster k, and |Sk| is the size of the cluster.

[0037] S3: Extract the hierarchical structure, layer type, and number of neurons in each layer of the model to obtain the architecture information of the model.

[0038] Preferably, the difference between the weight matrix before and after fine-tuning is calculated, and the L2 norm distance measurement method is used; cluster analysis is performed on the weight matrix after fine-tuning to identify the newly introduced features during the fine-tuning process; the fine-tuned layer is marked and compared with the unfine-tuned layer to analyze its impact on the overall kinship of the model.

[0039] Furthermore, assuming that the weight matrix of the original model is W_pre and the weight matrix after fine-tuning is W_post, the L2 norm difference between them can be expressed as: Among them, ||·||_2 represents the L2 norm of the matrix; By calculating the value of ΔW, we can evaluate the overall impact of fine-tuning on the model weights. The larger the ΔW value, the greater the adjustment of the weights by fine-tuning.

[0040] After calculating the weight matrix differences, cluster analysis is performed on the fine-tuned weight matrix. A common method is to apply the K-means clustering algorithm to divide the eigenvectors of the weight matrix into K clusters. By analyzing the feature differences between different clusters, features related to model behavior are identified. The K-means algorithm divides by minimizing the sum of the squared distances between each data point and its cluster center. The specific formula is: Among them, x_i represents the data point, c_k represents the center of the kth cluster; By performing K-means clustering analysis on the weight matrix, the central features of each cluster can be obtained, and then the features related to the model behavior can be identified.

[0041] S4: By comparing the fingerprint information and feature information of the two AI models, the similarity of their architectures can be determined.

[0042] Preferably, the total number of layers L1 and L2 of the two models are compared, and if L1=L2, the architectures are considered similar; Compare the types of each layer, such as convolutional layer, fully connected layer, and pooling layer. If the types of corresponding layers are consistent, the architectures are further considered to be similar; For each layer, the number of neurons in the two models is compared. If the number of neurons in the two models is the same, the layer is considered similar; The similarity of the model architecture is quantified by defining a similarity metric function Sarch: Where L is the total number of layers of the two models, n 1,l and n 2,l are the number of neurons in the lth layer of model 1 and model 2 respectively. The closer the Sarch value is to 1, the more similar the two model architectures are. The weight similarity of the two models is further analyzed.

[0043] Furthermore, for the two models to be compared, the following architecture information is extracted: the type of each layer, such as convolutional layer, fully connected layer, pooling layer, etc. For each layer, the number of neurons is recorded, such as the number of output feature maps of the convolutional layer, the number of neurons of the fully connected layer, etc.; if the two models have the same number of layers, that is, L1=L2, then the architecture is considered to be similar in terms of the number of layers; if the number of layers is different, it is necessary to further analyze the degree of influence of the difference layer on the overall architecture; if the types of the corresponding layers are consistent, the architecture of the layer is considered to be similar in terms of the layer type; if there are inconsistencies in the layer types, these differences are recorded to evaluate their impact on the overall architecture similarity. For each layer, the number of neurons of the two models is compared. If the number of neurons of the corresponding layers is the same, then the layer is considered to be similar in terms of the number of neurons. If there is a difference, the degree of difference is calculated.

[0044] The value of S_arch ranges from 0 to 1. The closer the value is to 1, the more similar the architectures of the two models are. If S_arch is close to 1, it means that the two models are highly similar in the number of layers, layer types, and number of neurons in each layer, and have a direct blood relationship; if S_arch is lower, it means that the two models have significant differences in architecture, and the blood relationship is weak or non-existent.

[0045] S5: We further analyze whether the models are related by comparing the weight matrices of the corresponding layers of the two AI models.

[0046] Preferably, the weight similarity of the two models is compared layer by layer, and for the weight matrices W1 and W2 of the corresponding layers of the two models, the corresponding fingerprint feature vectors F1 and F2 are calculated by model fingerprint extraction, and the weight similarity matching is achieved by calculating the similarity between the two fingerprint feature vectors; The similarity is calculated using the cosine similarity method, which evaluates the similarity by calculating the cosine value of the angle between two vectors. The calculation formula is as follows: Among them, F 1i , F 2i Represent the components of F1 and F2 respectively. The value range of cosine similarity is between -1 and 1. When the cosine similarity is close to 1, it means that the two vectors are highly similar; when the cosine similarity is close to -1, it means that there is no similarity between the two vectors.

[0047] The similarity of the weight matrices corresponding to each layer of the two models is calculated in turn, and then the average is calculated to obtain the similarity of the two models for subsequent blood relationship analysis.

[0048] Furthermore, the total number of layers of the two models is obtained, recorded as L1 and L2. If L1≠L2, the model architectures are different. If L1= L2, proceed to the next step of comparison, and compare the layer types of the two models layer by layer (such as convolutional layer, fully connected layer, pooling layer, etc.). If the types of the corresponding layers are inconsistent, the model architectures are different. If the types of the corresponding layers are consistent, proceed to the next step of comparison, and for each layer, compare the number of neurons of the two models. If the number of neurons in the corresponding layers is different, the architecture of the layer is different.

[0049] If the number of neurons in corresponding layers is the same, the architecture of the layers is considered similar.

[0050] For each layer of each model, extract the weight matrix W and calculate the corresponding fingerprint feature vector F_layer. The specific steps are as follows: For each element Wi of the weight matrix W, apply the sigmoid activation function to normalize and map. The standardized weight elements are appropriately combined or counted to generate the fingerprint feature vector F_layer. The fingerprint feature vectors of all layers are spliced ​​into a long vector to form the high-dimensional feature vector F_model of the model.

[0051] The similarity of the weight matrices corresponding to each layer of the two models is calculated in turn, and then the average is calculated to obtain the overall similarity S_weight of the two models for subsequent blood relationship analysis.

[0052] Through the above steps, the architecture and weight similarity of the two models are systematically compared. The closer the values ​​of the model architecture similarity S_arch and weight similarity S_weight are to 1, the more similar the two models are in architecture and weight.

[0053] S6: Set a similarity threshold according to the weight matrix analysis structure of the corresponding layer, and determine whether there is a blood relationship through the similarity threshold.

[0054] Preferably, a similarity threshold is set according to the similarity calculation result, and the similarity between the two models is quantified by the similarity threshold, and whether they are related is determined accordingly. If the similarity is greater than the threshold, the models are considered to be related; The similarity threshold A is set in the following range: A>0.9 indicates high similarity: it means that the models are highly similar in structure, parameters or data; A between 0.7 and 0.9 indicates moderate similarity: it means that there is a certain degree of similarity between the models; A>0.5 indicates low similarity: it means that there is basic similarity between the models; Suppose we analyze two neural network models A and B to determine whether they are related. The similarity between the two models is calculated to be 0.85: If the similarity threshold is set to 0.8, then since 0.85>0.8, it is concluded that models A and B are related by blood; If the similarity is lower than the set threshold, for example, 0.65, then since 0.65<0.8, it is concluded that models A and B are not related by blood.

[0055] Further, set a similarity threshold: set a similarity threshold A according to specific needs; A> 0.9: high similarity, indicating that the models are highly similar in structure, parameters or data; 0.7 ≤ A ≤ 0.9: medium similarity, indicating that there is a certain degree of similarity between the models; A > 0.5: low similarity, indicating that there is basic similarity between the models; Combining the architecture similarity S_arch and weight similarity S_weight, if both are greater than the set threshold A, it is considered that the two models are related; Assume there are two models A and B, the specific analysis is as follows: The total number of layers of models A and B is 10 (L1=L2=10). Comparing layer by layer, it is found that the types of corresponding layers (such as convolutional layer, fully connected layer) are consistent, and the number of neurons in each layer is also the same. Therefore, the architecture similarity S_arch=1.

[0056] For each layer of the weight matrix, the fingerprint feature vector F is calculated, and the weight similarity of the corresponding layer is calculated using cosine similarity, assuming that the average similarity S_weight=0.85.

[0057] The similarity threshold A is set to 0.8. Since S_arch = 1>0.8, S_weight = 0.85>0.8, models A and B are considered to be related.

[0058] After introducing the method of the exemplary embodiment of the present invention, next, refer to Figure 2 A large-model blood relationship analysis system according to an exemplary embodiment of the present invention is described, and the system includes: Fingerprint feature extraction module, used to process the weight matrix of the target AI model and extract the fingerprint features of the model; Feature analysis module, used to further analyze the characteristics of the weight matrix of the AI ​​model; The model architecture acquisition module is used to extract the model's hierarchical structure, layer type, and number of neurons in each layer to obtain the model's architecture information; The architecture judgment module is used to judge the similarity of the architectures of two AI models by comparing the fingerprint information and feature information of the models; The weight matrix judgment module further analyzes whether the models are related by comparing the weight matrices of the corresponding layers of the two AI models; The threshold determination module is used to set a similarity threshold according to the weight matrix analysis structure of the corresponding layer, and determine whether there is a blood relationship through the similarity threshold.

[0059] After introducing the method and apparatus of the exemplary embodiment of the present invention, next, reference is made to Figure 3For a description of a computer-readable storage medium according to an exemplary embodiment of the present invention, please refer to Figure 3 , the computer-readable storage medium shown is a CD 30, on which a computer program (i.e., a program product) is stored. When the computer program is executed by the processor, it will implement the steps recorded in the above method implementation, for example, processing the weight matrix of the target AI model to extract the fingerprint features of the model; further analyzing the features of the weight matrix of the AI ​​model; extracting the hierarchical structure, layer type and number of neurons in each layer of the model to obtain the architecture information of the model; judging the similarity of their architectures by comparing the fingerprint information and feature information of the two AI models; further analyzing whether the models are related by blood by comparing the weight matrices of the corresponding layers of the two AI models; setting a similarity threshold according to the weight matrix analysis structure of the corresponding layer, and judging whether there is a blood relationship by the similarity threshold; the specific implementation method of each step will not be repeated here.

[0060] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which are not listed here one by one.

[0061] After introducing the method, apparatus and medium of the exemplary embodiments of the present invention, next, reference is made to Figure 4 A computing device for analyzing large model lineages according to an exemplary embodiment of the present invention.

[0062] Figure 4 A block diagram of an exemplary computing device 40 suitable for implementing embodiments of the present invention is shown, and the computing device 40 may be a computer system or a server. Figure 4 The computing device 40 shown is only an example and should not bring any limitation to the functionality and scope of use of the embodiments of the present invention.

[0063] like Figure 4 As shown, the components of the computing device 40 may include, but are not limited to: one or more processors or processing units 401 , a system memory 402 , and a bus 403 connecting different system components (including the system memory 402 and the processing unit 401 ).

[0064] The computing device 40 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computing device 40, including volatile and non-volatile media, removable and non-removable media.

[0065] The system memory 402 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 4021 and / or cache memory 4022. The computing device 40 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the ROM 4023 may be used to read and write non-removable, non-volatile magnetic media ( Figure 4 is not shown in the Figure 4 As shown in FIG. 4 , a disk drive for reading and writing a removable non-volatile disk (e.g., a “floppy disk”) and an optical disk drive for reading and writing a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 403 via one or more data medium interfaces. System memory 402 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of various embodiments of the present invention.

[0066] A program / utility 4025 having a set (at least one) of program modules 4024 may be stored, for example, in system memory 402, and such program modules 4024 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 4024 generally perform the functions and / or methods of the embodiments described herein.

[0067] The computing device 40 may also communicate with one or more external devices 404 (e.g., a keyboard, a pointing device, a display, etc.). Such communication may be performed via an input / output (I / O) interface 405. Furthermore, the computing device 40 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 406. Figure 4 As shown, the network adapter 406 communicates with other modules (such as the processing unit 401, etc.) of the computing device 40 via the bus 403. It should be understood that although Figure 4 Not shown, other hardware and / or software modules may be used in conjunction with computing device 40 .

[0068] The processing unit 401 executes various functional applications and data processing by running the program stored in the system memory 402, for example, processing the weight matrix of the target AI model to extract the fingerprint features of the model; further analyzing the features of the weight matrix of the AI ​​model; extracting the hierarchical structure, layer type and number of neurons in each layer of the model to obtain the architecture information of the model; judging the similarity of their architectures by comparing the fingerprint information and feature information of the two AI models; further analyzing whether the models are related by comparing the weight matrices of the corresponding layers of the two AI models; setting a similarity threshold according to the weight matrix analysis structure of the corresponding layer, and judging whether there is a blood relationship by the similarity threshold.

[0069] The specific implementation method of each step will not be repeated here. It should be noted that although several units / modules or sub-units / sub-modules of the synchronous escape wiring device based on multiple commodity flows are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more units / modules described above can be concretized in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided to be concretized by multiple units / modules.

[0070] In the description of the present invention, it should be noted that the terms “first”, “second” and “third” are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0071] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0072] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0073] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0074] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0075] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that can be executed by a processor. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., various media that can store program codes.

[0076] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

[0077] In addition, although the operations of the method of the present invention are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

Claims

1. A method for analyzing the blood relationship of a large model, characterized in that: include: Process the weight matrix of the target AI model to extract the fingerprint features of the model; Further analyze the characteristics of the weight matrix of the AI ​​model; Extract the model's hierarchical structure, layer type, and number of neurons in each layer to obtain the model's architecture information; By comparing the fingerprint information and feature information of two AI models, the similarity of their architectures can be determined; We then further analyze whether the models are related by comparing the weight matrices of the corresponding layers of the two AI models; The similarity threshold is set according to the weight matrix analysis structure of the corresponding layer, and whether there is a blood relationship is determined by the similarity threshold.

2. The method for analyzing the blood relationship of a large model as claimed in claim 1, characterized in that: The weight matrix of the target AI model is processed to extract the fingerprint features of the model, including: Assuming that the weight matrix of one layer of the model is W, its corresponding fingerprint Flayer can be calculated as follows: Among them, W i is the weight element in the weight matrix, n is the total number of elements in the weight matrix, and sigmoid is an activation function used to normalize and map the weights; Concatenate the features of all layers to convert the weight information of the entire model into a high-dimensional vector. If the model has multiple layers such as convolutional layers and fully connected layers, extract fingerprints from the weight matrix of each layer separately, and then concatenate the fingerprint features of each layer into a long vector to form a high-dimensional feature vector Fmodel, which is used to uniquely represent the model: Among them, L represents the number of layers of the large model, F i Represents the fingerprint of each layer of the large model.

3. The method for analyzing the blood relationship of a large model as claimed in claim 1, characterized in that: The weight matrix of the AI ​​model is further analyzed for its characteristics, including: Use the clustering algorithm to analyze the weight matrix, divide the eigenvectors of the weight matrix into K clusters, find out the feature differences between different clusters, and then identify the features related to the model behavior. The basic formula of the K-means algorithm is: Suppose there is a set of feature data (x1,x2,...,xn), where each data point , and select K cluster centers (c1,c2,...,cK), and divide them by minimizing the sum of the squared distances between each data point and its cluster center: Where I(ci=k) is the indicator function, its value is 1 when (ci=k), otherwise it is 0; In each iteration, the mean of each cluster is calculated and the data points are redistributed to the nearest cluster center until convergence. By performing K-means clustering analysis on the weight matrix, the central features of each cluster are obtained, and then the features related to the model behavior are identified.

4. The method for analyzing the blood relationship of a large model as claimed in claim 1, characterized in that: The hierarchical structure, layer type and number of neurons in each layer of the model are extracted to obtain the architecture information of the model, including: Calculate the difference between the weight matrix before and after fine-tuning, using the L2 norm distance measurement method; perform cluster analysis on the fine-tuned weight matrix to identify the newly introduced features during the fine-tuning process; mark the fine-tuned layer and compare it with the unfine-tuned layer to analyze its impact on the overall lineage of the model.

5. The method for analyzing the blood relationship of a large model as claimed in claim 1, characterized in that: The method of comparing the fingerprint information and feature information of the two AI models to determine the similarity of their architectures includes: Compare the total number of layers L1 and L2 of the two models. If L1=L2, the architectures are considered similar. Compare the types of each layer, such as convolutional layer, fully connected layer, and pooling layer. If the types of corresponding layers are consistent, the architectures are further considered to be similar; For each layer, the number of neurons in the two models is compared. If the number of neurons in the two models is the same, the layer is considered similar; The similarity of the model architecture is quantified by defining a similarity metric function Sarch: Among them, L is the total number of layers of the two models, n 1,l and n 2,l are the number of neurons in the lth layer of model 1 and model 2 respectively. The closer the Sarch value is to 1, the more similar the two model architectures are. The weight similarity of the two models is further analyzed.

6. The method for analyzing the blood relationship of a large model as claimed in claim 1, characterized in that: The weight matrices of the corresponding layers of the two AI models are then compared to further analyze whether the models are related: Compare the weight similarity of the two models layer by layer. For the weight matrices W1 and W2 of the corresponding layers of the two models, calculate the corresponding fingerprint feature vectors F1 and F2 through model fingerprint extraction, and achieve weight similarity matching by calculating the similarity between the two fingerprint feature vectors; The similarity is calculated using the cosine similarity method, which evaluates the similarity by calculating the cosine value of the angle between two vectors. The calculation formula is as follows: Among them, F 1i , F 2i Represent the components of F1 and F2 respectively. The value range of cosine similarity is between -1 and 1. When the cosine similarity is close to 1, it means that the two vectors are highly similar; when the cosine similarity is close to -1, it means that there is no similarity between the two vectors. The similarity of the weight matrices corresponding to each layer of the two models is calculated in turn, and then the average is calculated to obtain the similarity of the two models for subsequent blood relationship analysis.

7. The method for analyzing the blood relationship of a large model as claimed in claim 1, characterized in that: The similarity threshold is set according to the weight matrix analysis structure of the corresponding layer, and whether there is a blood relationship is determined by the similarity threshold, including: According to the similarity calculation results, a similarity threshold is set. The similarity between the two models is quantified by the similarity threshold, and it is judged whether they are related by blood. If the similarity is greater than the threshold, the models are considered to be related by blood. The similarity threshold A is set in the following range: A>0.9 indicates high similarity: it means that the models are highly similar in structure, parameters or data; A between 0.7 and 0.9 indicates moderate similarity: it means that there is a certain degree of similarity between the models; A > 0.5 indicates low similarity: it means that there is basic similarity between the models; Suppose we analyze two neural network models A and B to determine whether they are related. The similarity between the two models is calculated to be 0.85: If the similarity threshold is set to 0.8, then since 0.85>0.8, it is concluded that models A and B are related by blood; If the similarity is lower than the set threshold, for example, 0.65, then since 0.65<0.8, it is concluded that models A and B are not related by blood.

8. A system using the large model blood relationship analysis method as described in any one of claims 1 to 7, characterized in that: include: Fingerprint feature extraction module, used to process the weight matrix of the target AI model and extract the fingerprint features of the model; Feature analysis module, used to further analyze the characteristics of the weight matrix of the AI ​​model; The model architecture acquisition module is used to extract the model's hierarchical structure, layer type, and number of neurons in each layer to obtain the model's architecture information; The architecture judgment module is used to judge the similarity of the architectures of two AI models by comparing the fingerprint information and feature information of the models; The weight matrix judgment module further analyzes whether the models are related by comparing the weight matrices of the corresponding layers of the two AI models; The threshold determination module is used to set a similarity threshold according to the weight matrix analysis structure of the corresponding layer, and determine whether there is a blood relationship through the similarity threshold.

9. A computing device, comprising: at least one processor, memory, and input-output unit; The memory is used to store a computer program, and the processor is used to call the computer program stored in the memory to execute the steps of the large model bloodline analysis method as described in any one of claims 1 to 7.

10. A computer-readable storage medium comprising instructions, which, when executed on a computer, enables the computer to execute the steps of the large model kinship analysis method as described in any one of claims 1 to 7.

Images