Method, device, equipment, medium and product for determining multi-parameter correlation

Abstract

The present disclosure provides a method, device, equipment, medium and product for determining multi-parameter correlation, wherein the method for determining multi-parameter correlation includes: obtaining parameter invariants of multiple neural network models; splicing and encoding multiple parameter invariants based on a neural network encoder to obtain a parameter invariant vector; mapping the parameter invariant vector to a specified vector; converting the specified vector into a visual image based on an image generator, and the visual image is used to indicate the direct correlation of multiple neural network models. Through the embodiments of the present disclosure, while improving the reliability of determining model similarity, the intuitiveness of model similarity is improved.

Description

Background Art

[0002] At present, the existing large-model blood relationship determination scheme usually extracts the stress feedback characteristics of the large model, generates observation comparisons of stress data by inputting specific "allergens", and thus extracts the sensitive data characteristics of the large model, forms a test load to carry out safety detection on the large model, and then verifies whether there is a blood relationship between different large models.

[0003] However, the above solution requires a separate feature analysis for each large model, which is relatively complicated. In addition, large model parameters are the key features of large models, but parameters are easily affected by simple attacks such as weight value rearrangement. Although the parameters are changed, the model performance will not be affected, which increases the difficulty of determining the blood relationship of open source large models.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The purpose of the present disclosure is to provide a method, device, equipment, medium and product for determining multi-parameter correlation, which is used to overcome the problems of poor reliability of determining similarities between multiple large models due to limitations and defects of related technologies, at least to a certain extent.

[0006] According to a first aspect of an embodiment of the present disclosure, a method for determining multi-parameter correlation is provided, comprising: obtaining parameter invariants of multiple neural network models; concatenating and encoding the multiple parameter invariants based on a neural network encoder to obtain a parameter invariant vector; mapping the parameter invariant vector to a specified vector; and converting the specified vector into a visual image based on the image generator, wherein the visual image is used to indicate the direct correlation between the multiple neural network models.

[0007] In an exemplary embodiment of the present disclosure, obtaining parameter invariants of multiple neural network models includes:

[0008] Based on the corpora of the plurality of neural network models, counting the occurrence frequencies of sockets in the vocabulary corresponding to the corpora;

[0009] Screening the sockets according to a preset frequency and a preset size of the socket set, and generating a corresponding socket set according to the screening result;

[0010] Constructing a transformation matrix and a transposed matrix corresponding to the transformation matrix according to the size of the corpus and the size of the socket set;

[0011] Based on the transformation matrix and the transposed matrix, matrix conversion processing is performed on the parameter items of the neural network model to obtain the parameter invariant items.

[0012] In an exemplary embodiment of the present disclosure, performing matrix conversion processing on parameter items of the neural network model based on the transformation matrix and the transposed matrix includes:

[0013] Determine a calculation function corresponding to the encoder of the neural network model;

[0014] Extracting parameter items in the calculation function, and performing combined calculation on the parameter items according to preset calculation rules to obtain a combined parameter matrix;

[0015] The transformation matrix, the combined parameter matrix and the transposed matrix are multiplied in sequence to obtain the parameter invariant items.

[0016] In an exemplary embodiment of the present disclosure, concatenating and encoding a plurality of the parameter invariant items based on a neural network encoder to obtain a parameter invariant item vector includes:

[0017] According to the annotations or attribute features of the parameter invariants, multiple parameter invariants are concatenated into a vector of a specified length in a specific order, and the concatenated vector is encoded based on a neural network encoder to obtain the parameter invariant vector.

[0018] In an exemplary embodiment of the present disclosure, the designated vector is a Gaussian vector, and mapping the parameter invariant term vector to the designated vector includes:

[0019] The parameter invariant term vector is mapped to the Gaussian vector, where the Gaussian vector is used to indicate a distribution characteristic of the parameter invariant term vector.

[0020] In an exemplary embodiment of the present disclosure, the designated vector is a Gaussian vector, and converting the designated vector into a visual image based on the image generator includes:

[0021] The Gaussian vector is input into the trained image generator so that the image generator converts the Gaussian vector into the visual image, and the image generator includes an adversarial network and a variational autoencoder.

[0022] In an exemplary embodiment of the present disclosure, it further includes:

[0023] After obtaining the visualization image output by the image generator, determining the similarity of the plurality of visualization images according to a preset similarity;

[0024] Outputting a visualization image that meets the preset similarity requirement according to the result of the determination process.

[0025] According to a second aspect of an embodiment of the present disclosure, a device for determining a multi-parameter correlation is provided, comprising:

[0026] An acquisition module, configured to obtain parameter invariants of multiple neural network models;

[0027] A transformation module is configured to concatenate and encode a plurality of the parameter invariant items based on a neural network encoder to obtain a parameter invariant item vector;

[0028] A mapping module, configured to map the parameter invariant term vector to a specified vector;

[0029] A conversion module is configured to convert the specified vector into a visual image based on the image generator, wherein the visual image is used to indicate the direct correlation between the multiple neural network models.

[0030] According to a third aspect of the present disclosure, there is provided an electronic device, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute any one of the methods described above based on instructions stored in the memory.

[0031] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which a program is stored, and when the program is executed by a processor, the method for determining the multi-parameter correlation as described in any one of the above items is implemented.

[0032] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, characterized in that when the computer program is executed by a processor, the method for determining multi-parameter correlation as described in any one of the above is implemented.

[0033] Based on the disclosed embodiment, by obtaining the parameter invariants of multiple neural network models, and then splicing and encoding the multiple parameter invariants based on the neural network encoder to obtain the parameter invariant vector, the effect of parameter attack on the blood relationship judgment of the large model can be offset, and the invariants generated by all large models have the same size. Further, the parameter invariant vector is mapped to a specified vector, and finally, the specified vector is converted into a visual image based on the image generator. The visual image is used to indicate the direct correlation between the multiple neural network models, which is convenient for visual and intuitive judgment of model similarity.

[0034] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0036] Figure 1 A schematic diagram showing an exemplary system architecture to which a multi-parameter correlation determination solution according to an embodiment of the present invention can be applied;

[0037] Figure 2 is a flow chart of a method for determining multi-parameter correlation in an exemplary embodiment of the present disclosure;

[0038] Figure 3 is a flow chart of another method for determining multi-parameter correlation in an exemplary embodiment of the present disclosure;

[0039] Figure 4 is a flow chart of another method for determining multi-parameter correlation in an exemplary embodiment of the present disclosure;

[0040] Figure 5 is a flow chart of another method for determining multi-parameter correlation in an exemplary embodiment of the present disclosure;

[0041] Figure 6 is a flow chart of another method for determining multi-parameter correlation in an exemplary embodiment of the present disclosure;

[0042] Figure 7 is a flow chart of another method for determining multi-parameter correlation in an exemplary embodiment of the present disclosure;

[0043] Figure 8 is a flow chart of another method for determining multi-parameter correlation in an exemplary embodiment of the present disclosure;

[0044] Fig. 9 is an architectural diagram of a parameter invariant extraction model in another multi-parameter correlation determination solution in an exemplary embodiment of the present disclosure;

[0045] Fig.10 It is an architecture diagram of a large-model blood relationship determination system in another multi-parameter correlation determination solution in an exemplary embodiment of the present disclosure;

[0046] Fig.11 It is a flow chart of determining blood relationship between large models in another multi-parameter correlation determination scheme in an exemplary embodiment of the present disclosure;

[0047] Fig.12 is a block diagram of a device for determining multi-parameter correlation in an exemplary embodiment of the present disclosure;

[0048] Fig.13 is a block diagram of an electronic device in an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0049] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concepts of the example embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0050] In addition, the accompanying drawings are only schematic diagrams of the present disclosure, and the same reference numerals in the drawings represent the same or similar parts, so their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities, which do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0051] Figure 1 The diagram shows an exemplary system architecture to which a multi-parameter correlation determination solution according to an embodiment of the present invention can be applied.

[0052] like Figure 1 As shown, the system architecture 100 may include one or more of terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0053] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. According to the implementation requirements, there may be any number of terminal devices, networks and servers. For example, the server 105 may be a server cluster composed of multiple servers.

[0054] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Terminal devices 101, 102, 103 can be various electronic devices with display screens, including but not limited to smart phones, tablet computers, portable computers, desktop computers, etc.

[0055] In some embodiments, the method for determining the multi-parameter correlation provided in the embodiments of the present invention is generally performed by the server 105. Accordingly, the multi-parameter correlation determination device is generally arranged in the terminal device 103 (or the terminal device 101 or 102). In other embodiments, some terminals may have functions similar to those of the server device to perform the method.

[0056] The exemplary embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0057] Figure 2 Detailed description is a flow chart of a method for determining multi-parameter correlation in an exemplary embodiment of the present disclosure.

[0058] refer to Figure 2 , the method for determining the multi-parameter correlation may include:

[0059] Step S202, obtaining parameter invariants of multiple neural network models.

[0060] In an exemplary embodiment of the present disclosure, if the neural network model is used to process data in a specific format, then the data features that meet the format requirements can be used as parameter invariants. For example, for a neural network language model that accepts a fixed-dimensional vector as input, the dimension of the input vector is an invariant. In a speech recognition task, if the model requires an audio clip of a fixed length to be input, then the characteristics of the audio clip, such as the sampling rate and duration, are parameter invariants.

[0061] Step S204: concatenate and encode the plurality of parameter invariant items based on a neural network encoder to obtain a parameter invariant item vector.

[0062] In an exemplary embodiment of the present disclosure, multiple parameter invariants are spliced ​​together to comprehensively utilize multiple different features or attributes to improve the performance and generalization ability of the model. The splicing operation can connect multiple vectors in a specific order into a longer vector for input into subsequent processing modules.

[0063] In an exemplary embodiment of the present disclosure, assume that there are three parameter invariant vectors A, B and C, the length of each vector is n, and after splicing them together, the length of the new vector obtained is 3n, that is, [A1, A2,..., An, B1,B2,..., Bn, C1, C2,..., Cn].

[0064] Step S206, mapping the parameter invariant term vector to a specified vector.

[0065] In an exemplary embodiment of the present disclosure, by mapping the parameter invariant term vector to a specified vector, the data can have specific distribution characteristics, which is convenient for subsequent processing and analysis.

[0066] Step S208, based on the image generator, converting the specified vector into a visual image, wherein the visual image is used to indicate the direct correlation between the multiple neural network models.

[0067] Based on the disclosed embodiment, by obtaining the parameter invariants of multiple neural network models, and then splicing and encoding the multiple parameter invariants based on the neural network encoder to obtain the parameter invariant vector, the effect of parameter attack on the blood relationship judgment of the large model can be offset, and the invariants generated by all large models have the same size. Further, the parameter invariant vector is mapped to a specified vector, and finally, the specified vector is converted into a visual image based on the image generator. The visual image is used to indicate the direct correlation between the multiple neural network models, which is convenient for visual and intuitive judgment of model similarity.

[0068] Next, each step of the method for determining multi-parameter correlation is described in detail.

[0069] In an exemplary embodiment of the present disclosure, Figure 3 As shown, obtaining the parameter invariants of multiple neural network models includes:

[0070] Step S302: based on the corpora of the plurality of neural network models, counting the occurrence frequencies of sockets in the vocabulary corresponding to the corpora.

[0071] Step S304: screening the sockets according to the preset frequency and the preset size of the socket set, and generating a corresponding socket set according to the screening result.

[0072] Step S306: construct a transformation matrix and a transposed matrix corresponding to the transformation matrix according to the size of the corpus and the size of the socket set.

[0073] Step S308, performing matrix conversion processing on the parameter items of the neural network model based on the transformation matrix and the transposed matrix to obtain the parameter invariant items.

[0074] In an exemplary embodiment of the present disclosure, Figure 4 As shown, performing matrix conversion processing on parameter items of the neural network model based on the transformation matrix and the transposed matrix includes:

[0075] Step S402, determining a calculation function corresponding to the encoder of the neural network model.

[0076] Step S404: extract parameter items in the calculation function, and perform combination calculation on the parameter items according to preset calculation rules to obtain a combination parameter matrix.

[0077] Step S406, performing multiplication calculations on the transformation matrix, the combined parameter matrix and the transposed matrix in sequence to obtain the parameter invariant terms.

[0078] In an exemplary embodiment of the present disclosure, Figure 5 As shown, multiple parameter invariant items are concatenated and encoded based on a neural network encoder to obtain a parameter invariant item vector including:

[0079] Step S502, according to the annotations or attribute features of the parameter invariants, multiple parameter invariants are concatenated into a vector of a specified length in a specific order, and the concatenated vector is encoded based on a neural network encoder to obtain the parameter invariant vector.

[0080] In an exemplary embodiment of the present disclosure, Figure 6 As shown, the specified vector is a Gaussian vector, and mapping the parameter invariant term vector to the specified vector includes:

[0081] Step S602: Map the parameter invariant term vector to the Gaussian vector, where the Gaussian vector is used to indicate the distribution characteristics of the parameter invariant term vector.

[0082] In an exemplary embodiment of the present disclosure, a Gaussian vector is a vector with specific distribution characteristics, usually having a zero mean and a specific covariance matrix. By mapping the parameter invariant term vector to the Gaussian vector, the data can have specific distribution characteristics, which is convenient for subsequent processing and analysis.

[0083] In an exemplary embodiment of the present disclosure, Figure 7 As shown, the specified vector is a Gaussian vector, and converting the specified vector into a visual image based on the image generator includes:

[0084] Step S702: input the Gaussian vector to the trained image generator so that the image generator converts the Gaussian vector into the visual image, and the image generator includes an adversarial network and a variational autoencoder.

[0085] In an exemplary embodiment of the present disclosure, when a Gaussian vector is provided as input to an image generator, the image generator uses its learned mapping relationship to convert the Gaussian vector into a natural image. This process involves decoding and reconstructing the vector, as well as generating image details and textures.

[0086] In an exemplary embodiment of the present disclosure, in a generative adversarial network, a generator network receives a Gaussian vector as input and gradually generates an image through a series of convolution and deconvolution operations. The goal of the generator network is to generate natural images that are as realistic as possible to deceive the discriminator network. The discriminator network is responsible for determining whether the input image is a real natural image or an image generated by the generator. By continuously training the generator and discriminator networks, the generator can generate high-quality natural images.

[0087] In an exemplary embodiment of the present disclosure, Figure 8 As shown, the method for determining the multi-parameter correlation also includes:

[0088] Step S802: after obtaining the visualization image output by the image generator, determining the similarity of a plurality of the visualization images according to a preset similarity.

[0089] Step S804: outputting a visualization image that meets the preset similarity requirement according to the result of the determination process.

[0090] In an exemplary embodiment of the present disclosure, a parameter invariant term vector is input into a vector mapping module, mapped to a specific vector through a neural network encoder, and then the specific vector is input into an image generation module, and converted into a natural image using an image generator. This process can achieve the conversion from abstract parameter invariants to intuitive visual images, providing an effective means for data analysis and understanding.

[0091] Based on the above embodiments of the present disclosure, the basic model is identified by extracting the parameter invariant term, and mapped to a specific vector (such as a Gaussian vector) through a neural network encoder, and then the specific vector is read using an image generator and converted into an intuitive visual natural image. Similar parameter invariants will result in similar specific vectors, and finally similar images, so that the large model blood relationship can be intuitively and visually determined based on the image similarity.

[0092] In an exemplary embodiment of the present disclosure, Fig. 9As shown, a single encoder structure 900 in the Transformer neural network model has a general calculation function as shown in formula (1) and formula (2):

[0093] Formula (1)

[0094] Formula (2)

[0095] In an exemplary embodiment of the present disclosure, the specific explanations for the above formula (1) and formula (2) are as follows:

[0096] In the Transformer neural network model, there are two general calculation functions for each layer of the neural network, which include these expressions.

[0097] Attention calculation formula H’ Attn Included in this: W Q W K T 、W V W O The output formula of the n+1th layer is H’ n+1 Include W 1 W 2 .

[0098] in, W Q 、W K 、W V for The trainable weight parameter matrix, W O is the linear transformation matrix that processes them.

[0099] In the Transformer model, formula (1) represents a variant of the Multi-Head Self-Attention mechanism, where:

[0100] H n is the input of the nth layer; W Q is the weight matrix of the query, which will be input H nConvert into a query vector; W K is the weight matrix of the key, which will be input H n Convert to key direction; W V is the weight matrix of the value, which will be input H n Convert to a vector of values; W O is the output weight matrix, which transforms the output of multi-head attention into the final output vector; d is the dimension of the key vector. Here, we scale the dot product to prevent the gradient from disappearing or exploding. The softmax function converts the result of the dot product into a probability distribution so that the sum of the attention weights of each query vector to all key vectors is 1.

[0101] enter H n pass W Q , W k , W v Convert into query, key and value vectors respectively.

[0102] The result of weighted summation is passed through W o The calculation process of converting into the final output embodies the core of the self-attention mechanism in the Transformer model, which is to capture the dependencies within the sequence and generate new representations through the interaction of queries, keys, and values.

[0103] Common modifications to large model parameters include weight re-arrangement attacks, which change individual parameters but do not change the product of related parameters or the output of the large model, thus increasing the difficulty of determining the blood relationship of the large model.

[0104] Based on the standard corpus, this paper selects a set of tokens of appropriate size with the lowest frequency in the vocabulary, and constructs a matrix P (containing a sufficiently large set of tokens), whose transposed matrix is P T , use them to W Q W K T 、W V W O 、W 1 W 2 Mathematical changes are performed to obtain three invariants to offset parameter attacks and make the resulting invariants have the same size, universality and representativeness, while minimizing the impact of parameter updates in subsequent training processes. The three invariants A, B, and C are:

[0105]

[0106] H Attn The output of the self-attention layer is the input of the current layer H n The result after being processed by the self-attention mechanism; W 1 and W 2 are two linear transformation matrices in FFN, which are the learnable parameters of the model; b 1 and b 2 is with W 1 and W 2 The corresponding bias terms are also learnable parameters of the model; σ is the activation function.

[0107] Based on the above description, the meaning of formula (2) is:

[0108] First, the output of the self-attention layer H Attn By the first linear transformation W 1 , then add the bias b 1 .

[0109] Next, the above results are transformed nonlinearly through the activation function σ.

[0110] Then, the result of the activation function is transformed through a second linear transformation W 2 , and add the bias b 2 , and get the final output H n+1′ .

[0111] This FFN structure appears in each encoder and decoder layer, which allows the model to learn more complex representations independently for each position of the sequence after the self-attention mechanism. The design of this structure enables the model to capture the deep features in the sequence and introduces the expressive power of the model through non-linear activation functions.

[0112] Fig.10 Shown is the architecture diagram of the large-model blood relationship determination system.

[0113] In an exemplary embodiment of the present disclosure, Fig.10 As shown, the specific steps performed by the large model blood relationship determination system architecture 1000 are as follows:

[0114] In an exemplary embodiment of the present disclosure, a parameter invariant term extraction module analyzes and extracts parameter vectors of n large models to obtain n parameter invariant terms of the large models.

[0115] In an exemplary embodiment of the present disclosure, a neural network encoder (e.g., a convolutional encoder) in a vector mapping module takes parameter invariants as input, concatenates all parameter invariants of the model together, and maps them to n specific vectors (e.g., Gaussian vectors).

[0116] In an exemplary embodiment of the present disclosure, an image generator in an image generation module reads the specific vector and maps it into a natural image, thereby converting it into intuitive n visualization images.

[0117] In an exemplary embodiment of the present disclosure, similar parameter invariants will result in similar specific vectors and finally similar images, so that the blood relationship of the large model can be intuitively and visually determined based on the image similarity.

[0118] The specific process of determining the blood relationship between large models disclosed in this disclosure is as follows: Fig.11 shown.

[0119] In an exemplary embodiment of the present disclosure, Fig.11 As shown, the specific steps for determining the blood relationship between large models include:

[0120] Step 1: Input the large model parameters into the parameter invariant extraction module 1102 to obtain the parameter invariant vector.

[0121] Step 2: Input the parameter invariant term vector into the vector mapping module 1104, and concatenate all parameter invariant terms of the model together through a neural network encoder (such as a convolutional encoder) and map them to a specific vector (such as a Gaussian vector).

[0122] Step 3: Input the specific vector into the image generation module 1106, and use the image generator to map the Gaussian vector into a natural image and convert it into an intuitive visual image (the natural image can be set to various images such as plants and animals, and plants are taken as an example here).

[0123] Step 4: The visualized images generated by the large models to be compared are compared through the blood relationship judgment module 1108 to judge the image similarity. If they are similar, it is determined that there is a blood relationship, otherwise it is determined that there is no blood relationship.

[0124] Corresponding to the above method embodiments, the present disclosure also provides a device for determining multi-parameter correlation, which can be used to execute the above method embodiments.

[0125] Fig.12 It is a block diagram of a device for determining multi-parameter correlation in an exemplary embodiment of the present disclosure.

[0126] refer to Fig.12 , the multi-parameter correlation determination device 1200 may include:

[0127] The acquisition module 1202 is configured to obtain parameter invariants of multiple neural network models.

[0128] The transformation module 1204 is configured to concatenate and encode the plurality of parameter invariant items based on a neural network encoder to obtain a parameter invariant item vector.

[0129] The mapping module 1206 is configured to map the parameter invariant term vector to a specified vector.

[0130] The conversion module 1208 is configured to convert the specified vector into a visual image based on the image generator, and the visual image is used to indicate the direct correlation between the multiple neural network models.

[0131] In an exemplary embodiment of the present disclosure, the acquisition module 1202 is further configured to:

[0132] Based on the corpora of the plurality of neural network models, counting the occurrence frequencies of sockets in the vocabulary corresponding to the corpora;

[0133] Screening the sockets according to a preset frequency and a preset size of the socket set, and generating a corresponding socket set according to the screening result;

[0134] Constructing a transformation matrix and a transposed matrix corresponding to the transformation matrix according to the size of the corpus and the size of the socket set;

[0135] Based on the transformation matrix and the transposed matrix, matrix conversion processing is performed on the parameter items of the neural network model to obtain the parameter invariant items.

[0136] In an exemplary embodiment of the present disclosure, the conversion module 1208 is further configured to:

[0137] Determine a calculation function corresponding to the encoder of the neural network model;

[0138] Extracting parameter items in the calculation function, and performing combined calculation on the parameter items according to preset calculation rules to obtain a combined parameter matrix;

[0139] The transformation matrix, the combined parameter matrix and the transposed matrix are multiplied in sequence to obtain the parameter invariant items.

[0140] In an exemplary embodiment of the present disclosure, the transformation module 1204 is further configured to:

[0141] According to the annotations or attribute features of the parameter invariants, multiple parameter invariants are concatenated into a vector of a specified length in a specific order, and the concatenated vector is encoded based on a neural network encoder to obtain the parameter invariant vector.

[0142] In an exemplary embodiment of the present disclosure, the specified vector is a Gaussian vector, and the transformation module 1204 is further configured as:

[0143] The parameter invariant term vector is mapped to the Gaussian vector, where the Gaussian vector is used to indicate a distribution characteristic of the parameter invariant term vector.

[0144] In an exemplary embodiment of the present disclosure, the specified vector is a Gaussian vector, and the transformation module 1204 is further configured as:

[0145] The Gaussian vector is input into the trained image generator so that the image generator converts the Gaussian vector into the visual image, and the image generator includes an adversarial network and a variational autoencoder.

[0146] In an exemplary embodiment of the present disclosure, the multi-parameter correlation determination device 1200 is further configured to:

[0147] After obtaining the visualization image output by the image generator, determining the similarity of the plurality of visualization images according to a preset similarity;

[0148] Outputting a visualization image that meets the preset similarity requirement according to the result of the determination process.

[0149] Since the functions of the device 1200 have been described in detail in the corresponding method embodiments, the present disclosure will not elaborate on them here.

[0150] In summary, the technical effects of the embodiments provided by the present disclosure at least include:

[0151] (1) Select a sufficiently large feature matrix from the standard corpus to offset parameter attacks, and generate parameter invariants of the same size for large models with different parameter scales. The generated parameter invariants are universal and representative, and can offset the impact of parameter attacks on the blood relationship judgment of large models. It also makes the invariants generated by all large models have the same size, which is convenient for subsequent comparison.

[0152] (2) Map the specific vector generated by the neural network encoder to a natural image, thereby converting it into an intuitive visual image. The blood relationship of the large model is judged by image similarity. For natural images such as animals and plants, the human eye can judge the degree of similarity, and the judgment method is intuitive and convenient.

[0153] (3) Method for transforming parameter invariants into input vectors of the image generator: The generated large model parameter invariants are converted into vectors acceptable to the image generator through a neural network encoder, thereby facilitating the generation of natural images and visual intuitive judgment.

[0154] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0155] In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.

[0156] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as a system, method or program product. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as a "circuit", "module" or "system".

[0157] Refer to the following Fig.13 13 to 14. The electronic device 1300 according to this embodiment of the present invention is described. Fig.13 The electronic device 1300 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0158] like Fig.13As shown, the electronic device 1300 is in the form of a general computing device. The components of the electronic device 1300 may include but are not limited to: at least one processing unit 1310, at least one storage unit 1320, and a bus 1330 connecting different system components (including the storage unit 1320 and the processing unit 1310).

[0159] The storage unit stores program codes, which can be executed by the processing unit 1310, so that the processing unit 1310 performs the steps according to various exemplary embodiments of the present invention described in the above “Exemplary Method” section of this specification. For example, the processing unit 1310 can perform the method shown in the embodiment of the present disclosure.

[0160] The storage unit 1320 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 13201 and / or a cache storage unit 13202 , and may further include a read-only storage unit (ROM) 13203 .

[0161] The storage unit 1320 may also include a program / utility 13204 having a set (at least one) of program modules 13205, such program modules 13205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0162] Bus 1330 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0163] The electronic device 1300 may also communicate with one or more external devices 1340 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 1300, and / or may communicate with any device that enables the electronic device 1300 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 1350. In addition, the electronic device 1300 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 1360. As shown, the network adapter 1360 communicates with other modules of the electronic device 1300 via a bus 1330. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 1300, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0164] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0165] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible implementations, various aspects of the present invention can also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present invention described in the above "Exemplary Method" section of the present specification.

[0166] The program product for implementing the above method according to an embodiment of the present invention can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto, and in this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0167] Among them, the readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0168] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0169] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0170] In an exemplary embodiment of the present disclosure, a computer program product is also provided, which can be loaded or stored in any combination of one or more readable media, and can be written in any combination of one or more programming languages ​​to perform the program code of the present invention, wherein the programming language includes an object-oriented programming language, such as Java, C++, etc., and also includes a conventional procedural programming language, such as "C" language or similar programming language. The program code can be executed completely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on the remote computing device, or completely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0171] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to an exemplary embodiment of the present invention, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0172] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are to be considered exemplary only, and the true scope and concept of the present disclosure are indicated by the claims.

Claims

1. A method for determining multi-parameter correlation, characterized in that: include: Obtaining parameter invariants of multiple neural network models, wherein the neural network model is used to process data in a specific format, and data features that meet the requirements of the specific format are used as the parameter invariants, specifically including: Based on the corpora of the plurality of neural network models, counting the occurrence frequencies of sockets in the vocabulary corresponding to the corpora; Screening the sockets according to a preset frequency and a preset size of the socket set, and generating a corresponding socket set according to the screening result; Constructing a transformation matrix and a transposed matrix corresponding to the transformation matrix according to the size of the corpus and the size of the socket set; Performing matrix conversion processing on the parameter items of the neural network model based on the transformation matrix and the transposed matrix to obtain the parameter invariant items; splicing and encoding a plurality of the parameter invariant items based on a neural network encoder to obtain a parameter invariant item vector; Mapping the parameter invariant term vector to a specified vector; Based on an image generator, the designated vector is converted into a visual image, and the visual image is used to indicate the direct correlation between the plurality of neural network models.

2. The method for determining multi-parameter correlation according to claim 1, characterized in that: Performing matrix conversion processing on the parameter items of the neural network model based on the transformation matrix and the transposed matrix includes: Determine a calculation function corresponding to the encoder of the neural network model; Extracting parameter items in the calculation function, and performing combined calculation on the parameter items according to preset calculation rules to obtain a combined parameter matrix; The transformation matrix, the combined parameter matrix and the transposed matrix are multiplied in sequence to obtain the parameter invariant items.

3. The method for determining multi-parameter correlation according to claim 1, characterized in that: Based on the neural network encoder, multiple parameter invariant items are concatenated and encoded to obtain a parameter invariant item vector including: According to the annotations or attribute features of the parameter invariants, a plurality of the parameter invariants are concatenated into a vector of a specified length in a specific order, and the concatenated vector is encoded based on a neural network encoder to obtain the parameter invariant vector.

4. The method for determining multi-parameter correlation according to claim 1, characterized in that: The specified vector is a Gaussian vector, and mapping the parameter invariant term vector to the specified vector includes: The parameter invariant term vector is mapped to the Gaussian vector, where the Gaussian vector is used to indicate a distribution characteristic of the parameter invariant term vector.

5. The method for determining multi-parameter correlation according to claim 1, characterized in that: The specified vector is a Gaussian vector, and converting the specified vector into a visual image based on the image generator includes: The Gaussian vector is input into the trained image generator so that the image generator converts the Gaussian vector into the visual image, and the image generator includes an adversarial network and a variational autoencoder.

6. The method for determining multi-parameter correlation according to claim 1, characterized in that: Also includes: After obtaining the visualization image output by the image generator, determining the similarity of the plurality of visualization images according to a preset similarity; Outputting a visualization image that meets the preset similarity requirement according to the result of the determination process.

7. A device for determining multi-parameter correlation, characterized in that: include: An acquisition module is configured to acquire parameter invariants of multiple neural network models, wherein the neural network model is used to process data in a specific format, and data features that meet the requirements of the specific format are used as the parameter invariants, specifically including: Based on the corpora of the plurality of neural network models, counting the occurrence frequencies of sockets in the vocabulary corresponding to the corpora; Screening the sockets according to a preset frequency and a preset size of the socket set, and generating a corresponding socket set according to the screening result; Constructing a transformation matrix and a transposed matrix corresponding to the transformation matrix according to the size of the corpus and the size of the socket set; Performing matrix conversion processing on the parameter items of the neural network model based on the transformation matrix and the transposed matrix to obtain the parameter invariant items; A transformation module is configured to concatenate and encode a plurality of the parameter invariant items based on a neural network encoder to obtain a parameter invariant item vector; A mapping module, configured to map the parameter invariant term vector to a specified vector; A conversion module is configured to convert the specified vector into a visual image based on an image generator, wherein the visual image is used to indicate the direct correlation between the multiple neural network models.

8. An electronic device, characterized in that: include: Memory; as well as A processor coupled to the memory, wherein the processor is configured to execute the method for determining the multi-parameter correlation according to any one of claims 1 to 6 based on instructions stored in the memory.

9. A computer-readable storage medium having a program stored thereon, wherein when the program is executed by a processor, the method for determining the multi-parameter correlation according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for determining the multi-parameter correlation according to any one of claims 1 to 6 is implemented.

Images