Electronic data trust strength authentication hierarchical identification method, device, system and medium

Abstract

The application relates to the technical field of artificial intelligence, and provides a hierarchical identification method, device and system for electronic data trust strength authentication and a medium.The implementation scheme is as follows: a teacher end model is used to extract features of a sample index sequence to obtain a first feature vector; a student end model is used to extract features of the sample index sequence to obtain a second feature vector; the second feature vector and the first feature vector are integrated to obtain a multi-teacher distillation loss; a global feature distillation loss is determined based on the Euclidean distance square of the first feature vector and the second feature vector; the first feature vector is subjected to regularization processing to obtain a regularization loss; a mixed loss function is constructed based on the multi-teacher distillation loss, the global feature distillation loss and the regularization loss, the student end model is trained through the gradient of the mixed loss function, and the target student end model is used to perform hierarchical identification on electronic data according to trust strength.The application can improve the accuracy of the hierarchical identification result of the electronic data.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a graded identification method, apparatus, system and medium for electronic data trust strength authentication. Background Technology

[0002] Electronic data is widely used in many fields such as finance, energy, and healthcare. Classifying the trust level of electronic data is an important way to improve the efficiency of electronic data management and ensure the reliability of data use. It can play a key role in all aspects of data collection, transmission, storage and sharing.

[0003] Traditional large-scale models based on deep neural networks are often used for the graded identification of electronic data due to their certain feature extraction and semantic modeling capabilities. However, these models typically have a large number of parameters, and the training and inference processes require a lot of computing and storage resources, making them difficult to deploy in resource-constrained terminal devices or application scenarios with high real-time requirements.

[0004] Furthermore, while lightweight models offer advantages in computational overhead and inference speed, enabling rapid deployment on the terminal side, their feature extraction capabilities are somewhat lacking. They struggle to effectively capture deep semantic information in complex electronic data, resulting in lower accuracy of the classification and identification results. To ensure the reliability of data management, significant human and material resources are often required to manually correct the classification results of lightweight models, thereby reducing the efficiency of electronic data management. Summary of the Invention

[0005] This invention provides a graded identification method, apparatus, system, and medium for electronic data trust strength authentication, which can solve at least one of the above-mentioned technical problems.

[0006] In a first aspect, embodiments of the present invention provide a graded identification method for electronic data trust strength authentication, comprising:

[0007] The original electronic data and the preset prompt word classification template are processed to obtain the sample index sequence;

[0008] Multiple teacher-side models are used to perform deep semantic feature extraction on the sample index sequence to obtain multiple first feature vectors; and a student-side model is used to perform shallow semantic feature extraction on the sample index sequence to obtain a second feature vector.

[0009] Based on the first teacher weights in each of the teacher-side models, the second feature vector and each of the first feature vectors are integrated to obtain the multi-teacher distillation loss;

[0010] The global feature distillation loss is determined based on the squared Euclidean distance between each of the first feature vectors and the second feature vectors.

[0011] The regularization loss is obtained by applying normalization coefficients to each of the first feature vectors.

[0012] Based on the task-level loss, the multi-teacher distillation loss, the global feature distillation loss, and the regularization loss, a hybrid loss function is constructed. The student-side model is trained using the gradient of the hybrid loss function to obtain the target student-side model. The target student-side model is then used to perform trust strength level identification on the electronic data to be identified.

[0013] In one embodiment, the process of processing the original electronic data and the preset prompt word classification template to obtain a sample index sequence includes:

[0014] The original electronic data and the preset prompt word classification template are concatenated to obtain the concatenated text;

[0015] The concatenated text is segmented to obtain multiple sub-words;

[0016] By using a pre-defined vocabulary, each of the sub-words is mapped to obtain an integer index corresponding to each sub-word;

[0017] The sample index sequence is determined based on the integer index corresponding to each of the sub-words.

[0018] In one implementation, the step of performing deep semantic feature extraction on the sample index sequence using multiple teacher-side models to obtain multiple first feature vectors; and performing shallow semantic feature extraction on the sample index sequence using a student-side model to obtain a second feature vector, includes:

[0019] Based on the embedding layer parameters in the teacher-side model, each sample index in the sample index sequence is mapped to obtain the first word embedding matrix;

[0020] Based on the encoding layer parameters in the teacher-side model, the first word embedding matrix is ​​encoded to obtain deep semantic sequence features;

[0021] The deep semantic sequence features are subjected to mean pooling to obtain the first feature vector;

[0022] Based on the embedding layer parameters in the student-side model, each sample index in the sample index sequence is mapped to obtain the second word embedding matrix.

[0023] Based on the encoding layer parameters in the student-side model, the second word embedding matrix is ​​encoded to obtain shallow semantic sequence features;

[0024] The shallow semantic sequence features are subjected to mean pooling to obtain the second feature vector.

[0025] In one implementation, the step of integrating the second feature vector and each of the first feature vectors based on the first teacher weights in each of the teacher-side models to obtain the multi-teacher distillation loss includes:

[0026] The second feature vector and temperature coefficient are softened by an activation function to obtain the first probability distribution;

[0027] For each of the first feature vectors, perform the following loss calculation:

[0028] The activation function is used to soften the first feature vector and the temperature coefficient to obtain a second probability distribution;

[0029] The relative entropy of the first probability distribution and the second probability distribution is calculated to obtain the first divergence value;

[0030] Based on the mean of the first probability distribution and the second probability distribution, a third probability distribution is determined; for the first probability distribution and the second probability distribution, relative entropy is calculated with the third probability distribution to obtain a second divergence value and a third divergence value.

[0031] The fourth divergence value is determined based on the mean of the second and third divergence values;

[0032] The first divergence value and the fourth divergence value are weighted and summed based on the weighting coefficients to obtain the combined loss term;

[0033] When the loss calculation is performed on each of the first feature vectors, each first calculation result is determined based on the product of each of the combined loss terms and the temperature square factor; based on the first teacher weight in each of the teacher-side models, each of the first calculation results is weighted and summed to obtain the multi-teacher distillation loss.

[0034] In one implementation, determining the global feature distillation loss based on the squared Euclidean distance between each of the first feature vectors and the second feature vectors includes:

[0035] For each of the first feature vectors, the first feature distillation loss is determined based on the squared Euclidean distance between the first feature vector and the second feature vector;

[0036] Based on the second teacher weights in multiple teacher-side models, the first feature distillation losses are weighted and summed to obtain the global feature distillation loss.

[0037] In one implementation, the step of regularizing each of the first feature vectors using normalization coefficients to obtain a regularization loss includes:

[0038] By combining teachers from multiple teacher-side models, multiple teacher pairs are obtained;

[0039] For multiple teacher pairs, the second feature distillation loss of the teacher pair is determined based on the squared Euclidean distance between the first feature vector corresponding to the first teacher in the teacher pair and the first feature vector corresponding to the second teacher in the teacher pair.

[0040] The regularization loss is determined by multiplying the sum of the second feature distillation losses of each teacher pair with the normalization coefficient.

[0041] In one implementation, the step of constructing a hybrid loss function based on task-level loss, multi-teacher distillation loss, global feature distillation loss, and regularization loss, and training the student-side model using the gradient of the hybrid loss function to obtain the target student-side model, includes:

[0042] The student-side model is used to predict the original electronic data to obtain the predicted grading result corresponding to the original electronic data;

[0043] The task classification loss is obtained by calculating the cross-entropy between the predicted classification result corresponding to the original electronic data and the label corresponding to the original electronic data using the cross-entropy loss function.

[0044] Based on the task-level loss, the multi-teacher distillation loss, the global feature distillation loss, and the regularization loss, as well as the loss weights corresponding to each loss, the hybrid loss function is determined.

[0045] The gradient of the hybrid loss function is calculated using the backpropagation algorithm to obtain the gradient value;

[0046] If the gradient value is greater than a preset gradient threshold, the gradient value is clipped to obtain a clipped gradient value. The model parameters in the student-side model are then updated based on the clipped gradient value to train the student-side model and obtain the target student-side model. The model parameters in the student-side model include embedding layer parameters, encoding layer parameters, and classification layer parameters.

[0047] Secondly, embodiments of the present invention provide a graded identification device for electronic data trust strength authentication, comprising:

[0048] The processing module is used to process the raw electronic data and the preset prompt word classification template to obtain the sample index sequence;

[0049] The semantic feature extraction module is used to perform deep semantic feature extraction on the sample index sequence through multiple teacher-side models to obtain multiple first feature vectors; and to perform shallow semantic feature extraction on the sample index sequence through a student-side model to obtain second feature vectors.

[0050] An integration module is used to integrate the second feature vector and each of the first feature vectors based on the first teacher weights in each of the teacher-side models to obtain a multi-teacher distillation loss.

[0051] The global feature distillation loss determination module is used to determine the global feature distillation loss based on the squared Euclidean distance between each of the first feature vectors and the second feature vectors.

[0052] The regularization module is used to perform regularization processing on each of the first feature vectors using normalization coefficients to obtain the regularization loss;

[0053] The hierarchical identification module is used to construct a hybrid loss function based on task hierarchical loss, the multi-teacher distillation loss, the global feature distillation loss, and the regularization loss. The student-side model is trained using the gradient of the hybrid loss function to obtain a target student-side model. The target student-side model is then used to perform trust strength hierarchical identification on the electronic data to be identified.

[0054] In one embodiment, the processing module includes:

[0055] The splicing unit is used to splice the original electronic data and the preset prompt word hierarchical template to obtain spliced ​​text;

[0056] The word segmentation unit is used to segment the concatenated text into multiple sub-words;

[0057] The first mapping unit is used to map each of the sub-words using a preset vocabulary to obtain the integer index corresponding to each of the sub-words;

[0058] The sample index sequence determination unit is used to determine the sample index sequence based on the integer index corresponding to each of the sub-words.

[0059] In one embodiment, the semantic feature extraction module includes:

[0060] The second mapping unit is used to map each sample index in the sample index sequence based on the embedding layer parameters in the teacher-side model to obtain the first word embedding matrix;

[0061] The first encoding unit is used to encode the first word embedding matrix based on the encoding layer parameters in the teacher-side model to obtain deep semantic sequence features;

[0062] The first mean pooling unit is used to perform mean pooling on the deep semantic sequence features to obtain the first feature vector;

[0063] The third mapping unit is used to map each sample index in the sample index sequence based on the embedding layer parameters in the student-side model to obtain the second word embedding matrix;

[0064] The second encoding unit is used to encode the second word embedding matrix based on the encoding layer parameters in the student-side model to obtain shallow semantic sequence features;

[0065] The second mean pooling unit is used to perform mean pooling on the shallow semantic sequence features to obtain the second feature vector.

[0066] In one embodiment, the integration module includes:

[0067] A softening processing unit is used to soften the second feature vector and temperature coefficient through an activation function to obtain a first probability distribution;

[0068] The loss calculation unit is configured to perform the following loss calculation for each of the first feature vectors:

[0069] A softening subunit is used to soften the first feature vector and the temperature coefficient through the activation function to obtain a second probability distribution;

[0070] The first relative entropy calculation subunit is used to calculate the relative entropy of the first probability distribution and the second probability distribution to obtain the first divergence value;

[0071] The second relative entropy calculation subunit is used to determine the third probability distribution based on the mean of the first probability distribution and the second probability distribution; and to calculate the relative entropy between the first probability distribution and the second probability distribution and the third probability distribution to obtain the second divergence value and the third divergence value.

[0072] The fourth divergence value determination sub-unit is used to determine the fourth divergence value based on the mean of the second and third divergence values;

[0073] The first weighted summation subunit is used to perform a weighted summation of the first divergence value and the fourth divergence value based on the weight coefficients to obtain the combined loss term;

[0074] The second weighted summation subunit is used to determine each first calculation result based on the product of each combined loss term and the temperature square factor when the loss calculation is performed on each of the first feature vectors; and to perform weighted summation on each of the first calculation results based on the first teacher weight in each of the teacher-side models to obtain the multi-teacher distillation loss.

[0075] In one embodiment, the global feature distillation loss determination module includes:

[0076] The first feature distillation loss unit is used to determine the first feature distillation loss for each of the first feature vectors based on the squared Euclidean distance between the first feature vector and the second feature vector.

[0077] The weighted summation unit is used to perform weighted summation on each of the first feature distillation losses based on the second teacher weights in the multiple teacher-side models, to obtain the global feature distillation loss.

[0078] In one implementation, the regularization processing module includes:

[0079] The combination unit is used to combine teachers in multiple teacher-side models to obtain multiple teacher pairs;

[0080] The second feature distillation loss unit is used to determine the second feature distillation loss of a teacher pair based on the squared Euclidean distance between the first feature vector corresponding to the first teacher in the teacher pair and the first feature vector corresponding to the second teacher in the teacher pair.

[0081] A regularization loss unit is used to determine the regularization loss based on the sum of the second feature distillation losses of each teacher pair and the product of the normalization coefficient.

[0082] In one embodiment, the hierarchical identification module includes:

[0083] The first prediction unit is used to predict the original electronic data through the student-side model to obtain the prediction grading result corresponding to the original electronic data.

[0084] The cross-entropy calculation unit is used to calculate the cross-entropy between the predicted classification result corresponding to the original electronic data and the label corresponding to the original electronic data through the cross-entropy loss function, so as to obtain the task classification loss.

[0085] The hybrid loss function determination unit is used to determine the hybrid loss function based on the task-level loss, the multi-teacher distillation loss, the global feature distillation loss, and the regularization loss, as well as the loss weights corresponding to each loss.

[0086] The gradient calculation unit is used to calculate the gradient of the hybrid loss function using the backpropagation algorithm to obtain the gradient value;

[0087] The student-side model training unit is used to clip the gradient value when the gradient value is greater than a preset gradient threshold, obtain the clipped gradient value, and update the model parameters in the student-side model based on the clipped gradient value to train the student-side model. The model parameters in the student-side model include embedding layer parameters, encoding layer parameters, and classification layer parameters.

[0088] Thirdly, embodiments of the present invention also provide a graded identification system for electronic data trust strength authentication, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method described in any one of the embodiments of the present invention.

[0089] Fourthly, embodiments of the present invention also provide a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to perform the method described in any one of the embodiments of the present invention.

[0090] The technical solution of this invention processes the original electronic data and a preset prompt word hierarchy template. The resulting sample index sequence serves as the input for both the teacher-side and student-side models, meaning they share the same prompt word hierarchy template. This ensures that the student-side model can efficiently learn the deep semantic features of the teacher-side model during knowledge learning. Subsequently, the teacher-side and student-side models extract features from the sample index sequence, fully leveraging the semantic capture capabilities of the teacher-side model and the lightweight advantages of the student-side model, resulting in a first feature vector containing deep semantic relationships and a second feature vector containing shallow semantic relationships.

[0091] Based on this, firstly, by integrating the first and second feature vectors, the resulting multi-teacher distillation loss possesses more comprehensive knowledge, laying a foundation of deep semantic knowledge for subsequent training of the student-side model. Secondly, the squared Euclidean distance between each of the first and second feature vectors serves as a global feature constraint, reducing the differences in global semantic representation between the student-side and teacher-side models, thereby enhancing the overall feature expression capability of the student-side model. Thirdly, the regularization loss, as an expert consistency constraint, balances the knowledge contributions of each teacher-side model, reducing the risk of inaccurate classification results due to the student-side model's over-reliance on a single teacher model. Finally, a hybrid loss function is constructed based on the task classification loss, multi-teacher distillation loss, global feature distillation loss, and regularization loss, and the student-side model is updated based on the gradient of the hybrid loss function to obtain the target student-side model. Because this target student-side model integrates deep semantic knowledge, global feature constraints, and expert consistency constraints during knowledge distillation training, it can maintain low computational complexity while possessing high feature expression capability and prediction accuracy when performing trust strength classification recognition tasks on the electronic data to be identified, thus effectively improving data management efficiency.

[0092] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0093] The accompanying drawings are provided for a better understanding of this solution and do not constitute a limitation of the invention. Wherein:

[0094] Figure 1 This is a flowchart of a graded identification method for electronic data trust strength authentication according to an embodiment of the present invention;

[0095] Figure 2 This is a structural block diagram of a graded identification device for electronic data trust strength authentication according to an embodiment of the present invention;

[0096] Figure 3 This is a schematic block diagram of an electronic device used to implement the methods of embodiments of the present invention. Detailed Implementation

[0097] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0098] Figure 1 This is a flowchart of a graded identification method for electronic data trust strength authentication according to an embodiment of the present invention.

[0099] like Figure 1 As shown, the hierarchical identification method for electronic data trust strength authentication may include:

[0100] S110, Process the original electronic data and the preset prompt word classification template to obtain the sample index sequence;

[0101] S120: Deep semantic feature extraction is performed on the sample index sequence through multiple teacher-side models to obtain multiple first feature vectors; and shallow semantic feature extraction is performed on the sample index sequence through the student-side model to obtain second feature vectors.

[0102] S130, based on the first teacher weights in each teacher-side model, integrates the second feature vector and each first feature vector to obtain the multi-teacher distillation loss;

[0103] S140, Determine the global feature distillation loss based on the squared Euclidean distance between each first feature vector and the second feature vector;

[0104] S150, regularization is performed on each first feature vector by normalization coefficients to obtain regularization loss;

[0105] S160 constructs a hybrid loss function based on task-level loss, multi-teacher distillation loss, global feature distillation loss, and regularization loss. The student-side model is trained using the gradient of the hybrid loss function to obtain the target student-side model. The target student-side model is then used to perform trust strength level identification on the electronic data to be identified.

[0106] For example, in this embodiment of the invention, the electronic data to be identified is classified into levels based on the trust strength of the electronic data. Trust strength refers to the degree to which the electronic data needs to be trusted. The classification result obtained after classifying the electronic data to be identified using a model is the trust strength value. The higher the trust strength value, the greater the need for trust from a wider range of or more types of subjects. Trust strength classification refers to dividing the degree to which the electronic data needs to be trusted into levels.

[0107] For example, raw electronic data refers to data in the power supply chain that has not yet been cleaned, transformed, or labeled. It can be in the form of structured database records (such as transaction flow data, contract documents in the power supply chain, and equipment inspection records) or unstructured data (such as text, images, log files, etc.).

[0108] For example, the original electronic data can come from a database or sensor in the business system, or it can be received in real time from electronic documents uploaded by users through an interface.

[0109] For example, the prompt word grading template is a set of prompt words engineering proposed based on the expert scoring experience in the field of power material supply chain. It includes an expert scoring experience table and a trust strength classification basis table to ensure the interpretability and consistency of the grading process.

[0110] For example, the expert scoring experience table is shown in Table 1, and the trust strength classification level is shown in Table 2.

[0111] Table 1

[0112]

[0113] Table 2

[0114]

[0115] For example, a sample index sequence refers to an integer index sequence of discrete tokens (words) obtained after parsing the original electronic data and the prompt word hierarchy template.

[0116] For example, the teacher-side model can be a large-scale language model used to extract deep semantic features from electronic data.

[0117] For example, the teacher-side model could be a neural network model based on an attention mechanism, such as the Transformer.

[0118] For example, the first feature vector refers to the deep semantic representation obtained by the teacher-side model after encoding the sample index sequence and then performing mean pooling, which preserves high-dimensional semantic information.

[0119] For example, the student-side model can be a lightweight Transformer model or a convolutional neural network model, used to extract shallow semantic features while ensuring inference speed.

[0120] For example, the second feature vector refers to the shallow semantic representation obtained by the student-side model after encoding the sample index sequence and then performing mean pooling, which represents lightweight semantic features.

[0121] For example, multi-teacher distillation loss refers to a loss function that, during the knowledge distillation process, weights and integrates the feature vectors of the student-side model based on the output feature vectors of each teacher-side model and their corresponding teacher weights, thereby measuring the difference in feature representation between the student-side model and multiple teacher-side models.

[0122] For example, global feature distillation loss refers to a loss function that measures the degree of global consistency between the first feature vector output by the teacher-side model and the second feature vector output by the student-side model in the feature space during the knowledge distillation process, based on the overall feature difference between them. This loss is achieved by using the squared Euclidean distance.

[0123] For example, regularized loss refers to a custom loss term obtained by regularizing the first feature vector of each teacher-side model through normalization coefficients. It is used to balance the knowledge contributions of different teacher-side models and avoid overfitting to a single teacher feature.

[0124] For example, the hybrid loss function is a comprehensive loss or objective function constructed based on task-level loss, multi-teacher distillation loss, global feature distillation loss, and regularization loss, used to jointly optimize the student-side model.

[0125] For example, the target student-side model refers to the model trained by backpropagation under the constraints of the above-mentioned hybrid loss function, which can achieve high-precision hierarchical identification of the trust strength of electronic data while ensuring inference speed.

[0126] According to the above implementation method, this invention extracts multiple sample indices from the original electronic data and a preset prompt word classification template to obtain a sample index sequence; it then extracts deep features from the sample index sequence using multiple teacher-side models to achieve the ability to capture complex semantics; and extracts shallow features using a student-side model to achieve lightweight reasoning. Based on this, the knowledge from the teacher-side model is distilled into the student-side model using a first feature vector obtained from deep feature extraction and a second feature vector obtained from shallow feature extraction. Specifically, multi-teacher distillation loss and global feature distillation loss guide the student model to consider both local semantics and global representation; regularization loss ensures balanced utilization of knowledge from different teachers. Thus, during the knowledge transfer process, the student-side model is trained using a hybrid loss function constructed based on task classification loss, multi-teacher distillation loss, global feature distillation loss, and regularization loss. The resulting target student-side model not only significantly improves the accuracy of trust strength classification but also effectively reduces the cost of manual correction, thereby improving the overall efficiency of electronic data management.

[0127] In one implementation, the original electronic data and a preset prompt word hierarchy template are processed to obtain a sample index sequence, including: concatenating the original electronic data and the preset prompt word hierarchy template to obtain concatenated text; segmenting the concatenated text to obtain multiple sub-words; mapping each sub-word through a preset vocabulary to obtain an integer index corresponding to each sub-word; and determining the sample index sequence based on the integer index corresponding to each sub-word.

[0128] For example, spliced ​​text refers to text obtained by splicing raw electronic data and prompt word classification templates.

[0129] For example, before splicing, the original electronic data is preprocessed (e.g., word segmentation and stop word removal, named entity recognition, deep learning model to extract core vocabulary, standardization or discretization of numerical fields, etc.) to extract key attributes and convert them into a structured format that is easy to splice later.

[0130] In this example, a structured format, such as JavaScript Object Notation (JSON), Extensible Markup Language (XML), or structured text segment format, is used.

[0131] It should be noted that during preprocessing, if the original electronic data contains both images and text, a unified text description can be extracted using a multimodal model.

[0132] For example, for key attributes in the original electronic data (e.g., historical quality records, on-time delivery rate), the corresponding evaluation criteria in the expert scoring experience table are looked up to evaluate the key attribute. The evaluation results obtained based on the evaluation criteria (e.g., high, medium, low) are mapped to the trust strength classification table, and then the corresponding expert tips are found.

[0133] For example, if a supplier's historical quality record shows "zero major quality incidents," this can be mapped to the "quality stability" evaluation criterion in the expert scoring experience table. Since the evaluation criterion "quality stability" corresponds to a "high" level, the expert prompts corresponding to the "high" level (such as "excellent quality performance, no problems occurred") are then searched. The extracted key attributes and the mapped expert prompts are dynamically filled into a preset prompt grading template. In this way, a structured, concatenated text rich in expert knowledge and grading information can be obtained.

[0134] For example, system prompts and text content can be concatenated into a single string using string concatenation operations (such as Python's "+" method);

[0135] For example, concatenation can also be achieved by replacing placeholders in a predefined template format (such as JSON or placeholders).

[0136] For example, considering the characteristics of text in the field of power materials, a specialized word segmenter (e.g., a word segmenter based on a dictionary, rules, or statistical model) can be selected or trained. This word segmenter is then used to segment the concatenated text, thereby obtaining multiple sub-words.

[0137] For example, a pre-trained word segmenter can segment "cable quality performance excellent" into "cable", "quality", "performance", and "excellent". Here, "cable", "quality", "performance", and "excellent" are all sub-words.

[0138] For example, a thesaurus containing specialized terminology, industry vocabulary, and general terms in the power supply chain is pre-established. Each subword in the thesaurus corresponds to a unique integer index. For each subword obtained from word segmentation in the aforementioned example, its corresponding integer index is searched in the pre-established thesaurus. In this way, a sample index sequence consisting of integer indices corresponding to multiple subwords can be obtained.

[0139] It should be noted that for subwords not in the vocabulary, they can be mapped to a special "unknown word" index. An "unknown word" index, for example, <unk>.

[0140] According to the above implementation method, firstly, the original electronic data is concatenated with a preset prompt word hierarchy template to obtain concatenated text that simultaneously contains task instructions and data information, thus providing a clear semantic context for subsequent modeling. Then, the concatenated text is segmented into multiple sub-words to break down continuous text content into finer-grained semantic units, improving the model's (teacher-side model and student-side model) ability to understand details. Finally, each sub-word is mapped using a preset vocabulary to obtain the corresponding integer index, thereby converting natural language into a computer-processable numerical representation.

[0141] In one implementation, deep semantic feature extraction is performed on the sample index sequence using multiple teacher-side models to obtain multiple first feature vectors; and shallow semantic feature extraction is performed on the sample index sequence using a student-side model to obtain a second feature vector. This includes: mapping each sample index in the sample index sequence based on the embedding layer parameters in the teacher-side model to obtain a first word embedding matrix; encoding the first word embedding matrix based on the encoding layer parameters in the teacher-side model to obtain deep semantic sequence features; performing mean pooling on the deep semantic sequence features to obtain a first feature vector; mapping each sample index in the sample index sequence based on the embedding layer parameters in the student-side model to obtain a second word embedding matrix; encoding the second word embedding matrix based on the encoding layer parameters in the student-side model to obtain shallow semantic sequence features; and performing mean pooling on the shallow semantic sequence features to obtain a second feature vector.

[0142] For example, the embedding layer parameters in the teacher-side model include the trainable parameter matrix of the word embedding layer.

[0143] In this example, the integer index sequence ,in, Let i be the integer index of the i-th token in the predefined vocabulary V. This represents the maximum sequence length. The word embedding layer uses a trainable parameter matrix. Index each token Mapped to d-dimensional word vectors , that is Thus, the word embedding matrix is ​​obtained. ,have Where L is the sequence length and d is the dimension of the hidden layer of the model. Indicates L rows The real matrix space of columns.

[0144] For example, the encoding layer parameters in the teacher-side model include the weight matrices corresponding to the query, key, and value.

[0145] In this example, the initial embedding The first word embedding matrix is ​​then fed into the first-layer Transformer encoder. After that, the... Layer encoder with the first -1st floor (i.e., the first) The output of the previous layer encoder As input, the algorithm is iterated 61 times to finally obtain the deep semantic sequence features. .

[0146] In this example, the complete computation of the Transformer encoder is as follows: First, the first word embedding matrix is ​​mapped to a query matrix (Q), a key matrix (K), and a value matrix (V), and then divided into multiple sub-matrices in the column direction according to the number of attention heads, with each sub-matrix corresponding to one attention head. For each token position in each sub-matrix, a rotation transformation is performed to introduce relative position information, where the rotation angle is determined by the dimension index and the token position. Subsequently, the rotated query sub-matrix, key sub-matrix, and value sub-matrix are input into the latent attention mechanism, where the key matrix and value sub-matrix are reduced to the latent dimension by a low-rank compression matrix, and then restored to the original dimension by the decompression matrix of each head, while the query sub-matrix retains its original size. Each head calculates the similarity by the dot product of the query and the decompressed key, and obtains the attention weight after normalization. The value sub-matrix is ​​then weighted and summed to obtain the output of each head. The outputs of all heads are concatenated, projected onto the output, added to the input residual, and then normalized by root mean square to complete the single-layer attention computation.

[0147] In some layers (such as layer 32), the normalized matrix further generates a gating matrix, which is used to calculate the weights of each expert feedforward network. Each expert feedforward network undergoes a linear transformation of first expanding and then shrinking the columns, followed by activation function processing. The resulting outputs are weighted, summed, and then added to the input residuals and normalized to enhance the model's ability to capture complex semantic patterns. The forward computation continues to the final layer (such as layer 61), where the output matrix is ​​averaged along the sequence length dimension to generate the first feature vector, providing the basic semantic representation for subsequent distillation classification and multi-path prediction.

[0148] For example, the embedding layer parameters in the student-side model include the trainable parameter matrix of the word embedding layer.

[0149] In this example, the student-side model can use a 32-layer Transformer decoder with 28 attention heads per layer and hidden dimensions. In the shallow semantic feature extraction stage of the student-side model, firstly, based on the embedding layer parameters in the student-side model, each token index in the discrete token sequence of the input text D is mapped to a corresponding d-dimensional vector, thus obtaining the second word embedding matrix. Each row corresponds to a vector representation of a token. This second word embedding matrix serves as the input to the encoding layer of the student-side model, carrying the basic semantic information of the input sequence.

[0150] In this example, the encoding layer parameters of the student-side model, including trainable weights for the query, key, and value matrices, are used to encode the second word embedding matrix layer by layer. In the first-layer Transformer encoder, the second word embedding matrix is ​​mapped to a query matrix (Q), a key matrix (K), and a value matrix (V), and information is exchanged through multi-head attention and group-query attention (GQA) mechanisms. Specifically, the query matrix is ​​split head-by-head, and the key and value matrices are split group-by-group, capturing features from different mapping subspaces. The position of each token is also incorporated through rotation transformations, integrating positional information into the attention calculation. Then, after residual connections and root mean square normalization, the output of the first layer is used as the input to the second layer, and this process is repeated until the 32nd layer. The output of the 32nd layer is used as the shallow semantic sequence feature, thus comprehensively expressing the contextual information and shallow semantic patterns of each token in the sequence.

[0151] After shallow semantic sequence features are generated, mean pooling is performed along the sequence length dimension to compress the sequence information and generate a second feature vector of fixed dimension. This vector can effectively reflect the overall shallow semantic representation of the input sequence and provide basic semantic information for subsequent distillation, hierarchical processing and multi-view knowledge alignment.

[0152] It should be noted that in the 16th layer of the Transformer backbone, a lightweight expert routing adapter can be inserted into the student-side model. This adapter achieves weighted fusion of multi-expert features through Top-2 sparse gating and a 1×1 convolutional bottleneck structure, thereby further enriching the expressive power of shallow semantic sequence features. In this way, the resulting second feature vector comprehensively reflects the shallow semantic information of the input sequence and provides a stable and interpretable foundational representation for subsequent feature distillation and task classification in the student-side model.

[0153] According to the above implementation method, firstly, based on the embedding layer parameters of the teacher-side model, the sample index sequence is mapped to a first word embedding matrix, and deep semantic sequence features are generated through the encoding layer. Then, a first feature vector is obtained through mean pooling. This vector can fully capture the high-order semantic information of the sequence, providing accurate teacher signals for knowledge distillation. Subsequently, based on the embedding layer parameters of the student-side model, the same sample index sequence is mapped to a second word embedding matrix, and shallow semantic sequence features are generated through the encoding layer and obtained through mean pooling to obtain a second feature vector. In this way, the second feature vector comprehensively reflects the basic semantic information of the sequence. In addition, by calculating and distilling the differences in teacher features, it helps the student model learn efficiently and converge quickly.

[0154] In one implementation, based on the first teacher weights in each teacher-side model, the second feature vector and each of the first feature vectors are integrated to obtain a multi-teacher distillation loss, including: softening the second feature vector and temperature coefficient using an activation function to obtain a first probability distribution; for each of the first feature vectors, performing the following loss calculation: softening the first feature vector and temperature coefficient using an activation function to obtain a second probability distribution; calculating the relative entropy of the first probability distribution and the second probability distribution to obtain a first divergence value; and determining the first divergence value based on the mean of the first probability distribution and the second probability distribution. The third probability distribution is used; relative entropy calculations are performed on the first and second probability distributions with the third probability distribution to obtain the second and third divergence values; the fourth divergence value is determined based on the mean of the second and third divergence values; the first and fourth divergence values ​​are weighted and summed based on the weight coefficients to obtain the combined loss term; with loss calculations performed on each first feature vector, each first calculation result is determined based on the product of each combined loss term and the temperature square factor; based on the first teacher weights in each teacher-side model, each first calculation result is weighted and summed to obtain the multi-teacher distillation loss.

[0155] For example, the activation function is the softmax function.

[0156] For example, the temperature coefficient can be set according to actual needs, such as 2 or 0.5. When the temperature coefficient is set to greater than 1, it can reduce the differences between the various probability distributions, making the probability distribution more uniform; when it is set to less than 1, it can amplify the differences between the various probability distributions, making the probability distribution more concentrated.

[0157] It should be noted that the same activation function and the same temperature coefficient were used for softening both the first and second feature vectors. Using different temperature coefficients or activation functions might lead to imbalances in the numerical range and gradient distribution of the two types of features, thus affecting training stability.

[0158] For example, the first probability distribution refers to the probability distribution obtained after softening the second feature vector and the temperature coefficient through an activation function, which is used to reflect the relative importance of the second feature vector in each category.

[0159] In this example, the calculation process of the first probability distribution can be represented by the following function expression: In the formula, Represents the first probability distribution; Indicates the activation function; This represents the second eigenvector; This represents the temperature coefficient.

[0160] For example, the second probability distribution refers to the probability distribution obtained after softening the first feature vector and the temperature coefficient through an activation function, which is used to characterize the category distribution of the first feature vector.

[0161] In this example, the calculation process of the second probability distribution can be represented by the following function expression:

[0162] In the formula, This represents the second probability distribution; Indicates the activation function; This represents the first eigenvector; This represents the temperature coefficient.

[0163] For example, the first divergence value refers to a metric obtained by calculating the relative entropy of the first probability distribution and the second probability distribution, and is used to measure the distributional difference between the two.

[0164] For example, the third probability distribution refers to the probability distribution determined based on the mean of the first probability distribution and the second probability distribution, and is used as an intermediate reference benchmark between the two distributions.

[0165] For example, the second divergence value refers to the calculated result of the relative entropy between the first probability distribution and the third probability distribution; the third divergence value refers to the calculated result of the relative entropy between the second probability distribution and the third probability distribution, and is used to jointly measure the degree of difference between the two distributions and the benchmark distribution.

[0166] For example, the fourth divergence value refers to the distribution difference measure determined based on the mean of the second and third divergence values, and is used to balance the differences between different distributions in loss calculation.

[0167] For example, the combined loss term refers to the loss value obtained by weighting and summing the first divergence value and the fourth divergence value based on the weight coefficients, which is used to comprehensively reflect the contribution of different divergence measures to training optimization.

[0168] In this example, taking a first feature vector and a second feature vector as examples, the calculation process of the combined loss term can be represented by the following function expression:

[0169] ;

[0170] ; ;

[0171] ;

[0172] In the formula, Indicates the first divergence value; Let represent the probability value of category i in the second probability distribution; i represents the category. This represents the probability value of category i in the first probability distribution; P represents the third probability distribution; Q represents the second probability distribution; and Q represents the first probability distribution. Indicates the fourth divergence value; Indicates the third divergence value; Indicates the second divergence value; Represents the combined loss term; This represents the weighting coefficient, which can be set to 0.7. This represents the weighting coefficient, which can be set to 0.3.

[0173] For example, the first calculation result refers to the calculation result determined based on the product of each combined loss term and the temperature square factor when loss calculation is performed for each first feature vector, and is used for subsequent teacher weighted summarization.

[0174] For example, the temperature square factor is the square of the temperature coefficient, which is the same as the temperature coefficient in the previous example.

[0175] For example, the first teacher weight in the teacher-side model refers to the weight coefficient assigned to the first teacher model among multiple teacher-side models. It is used to perform a weighted summation of the various first calculation results when calculating the multi-teacher distillation loss, thereby reflecting the contribution ratio of different teacher models to the distillation process. The larger the value of the first teacher weight, the more the "student" follows the "teacher".

[0176] According to the above implementation method, firstly, the second feature vector is softened using an activation function and a temperature coefficient to obtain a first probability distribution, which serves as the benchmark distribution for teacher-side knowledge. For each first feature vector, the same activation function and temperature coefficient are used to soften it, resulting in a second probability distribution, thus ensuring the comparability of the two distributions on the same scale. Subsequently, the relative entropy of the first and second probability distributions is calculated to obtain a first divergence value, used to measure the difference between the student and teacher sides. A third probability distribution, composed of the means of both, is further introduced, and its relative entropy is calculated with the first and second probability distributions respectively to obtain second and third divergence values. A fourth divergence value is then determined using the mean, achieving a balanced and robust measure of the difference. The first and fourth divergence values ​​are weighted and summed based on weighting coefficients to obtain a combined loss term. This combined loss term highlights the main differences while maintaining overall distribution consistency. When the above loss calculation is performed on all first feature vectors, each combined loss term is combined with a temperature squared factor to determine the corresponding first calculation result, thereby enhancing the regulatory effect of the temperature parameter on gradient scaling. Finally, based on the first teacher weights of multiple teacher-side models, the results of each first calculation are weighted and summed to obtain the multi-teacher distillation loss. Subsequently, this multi-teacher distillation loss is used as one of the losses for training the student-side model, enabling the student-side model to receive richer and more comprehensive guidance during the knowledge learning process.

[0177] In one implementation, the global feature distillation loss is determined based on the squared Euclidean distance between each first feature vector and the second feature vector, including: for each first feature vector, determining the first feature distillation loss based on the squared Euclidean distance between the first feature vector and the second feature vector; and weighting and summing each first feature distillation loss based on the second teacher weights in multiple teacher-side models to obtain the global feature distillation loss.

[0178] For example, the calculation process of global feature distillation loss can be represented by the following functional expression: .

[0179] In the formula, This represents the global feature distillation loss; K represents the total number of teacher-side models. This represents the weight of the second teacher in the teacher-side model k; This represents the first feature vector of the teacher-side model K; This represents the second feature vector of the student-side model; This represents the distillation loss of each primary characteristic.

[0180] For example, the second teacher weight in the teacher-side model is used to control the magnitude of the influence of different teachers on students. Its value can be the same as or different from the first teacher weight, and can be set according to actual needs.

[0181] According to the above implementation method, for each first feature vector, the squared Euclidean distance between it and the second feature vector is first calculated to determine the distillation loss of the first feature. This process can accurately measure the difference between the features of the student-side model and the features of the teacher-side model, promoting feature alignment. Subsequently, the distillation losses of all first features are weighted and summed by combining the second teacher weights in multiple teacher-side models to obtain the global feature distillation loss. This not only makes full use of the knowledge advantages of the multi-teacher-side models, but also provides comprehensive guidance for the features in the student-side model, improving the overall consistency and stability of the distillation effect.

[0182] In one implementation, regularization is performed on each first feature vector using a normalization coefficient to obtain a regularization loss, including: combining teachers in multiple teacher-side models to obtain multiple teacher pairs; for multiple teacher pairs, determining the second feature distillation loss of the teacher pair based on the squared Euclidean distance between the first feature vector corresponding to the first teacher in the teacher pair and the first feature vector corresponding to the second teacher in the teacher pair; and determining the regularization loss based on the sum of the second feature distillation losses of each teacher pair multiplied by the normalization coefficient.

[0183] For example, the combination can be carried out in a random selection manner, that is, randomly selecting two teachers from the teacher-side model as a teacher pair.

[0184] For example, the calculation process of regularization loss can be represented by the following functional expression: In the formula, Indicates the regularized loss; K represents the number of teacher pairs; Let i represent the first eigenvector corresponding to the first teacher i; This represents the first eigenvector corresponding to the second teacher j; This indicates the teacher's second characteristic distillation loss; This represents the normalization coefficient.

[0185] According to the above implementation method, firstly, teachers from multiple teacher-side models are combined to form multiple teacher pairs, thereby fully exploring the complementary information between different teacher-side models. Then, for each teacher pair, the squared Euclidean distance between the first feature vector corresponding to the first teacher and the first feature vector corresponding to the second teacher is calculated to determine the second feature distillation loss for that teacher pair. This operation helps to reflect the consistency differences in features among teachers and improves the accuracy of the distillation effect. Finally, the second feature distillation losses of all teacher pairs are summed and multiplied by the normalization coefficient to obtain the regularization loss. This step not only ensures that the contributions of different teacher pairs are reasonably balanced but also enhances the adaptability of the student model to the overall feature distribution.

[0186] In one implementation, a hybrid loss function is constructed based on task classification loss, multi-teacher distillation loss, global feature distillation loss, and regularization loss. The student-side model is trained using the gradient of the hybrid loss function to obtain the target student-side model. This includes: predicting the original electronic data using the student-side model to obtain the predicted classification result corresponding to the original electronic data; calculating the cross-entropy between the predicted classification result corresponding to the original electronic data and the label corresponding to the original electronic data using the cross-entropy loss function to obtain the task classification loss; determining the hybrid loss function based on the task classification loss, multi-teacher distillation loss, global feature distillation loss, and regularization loss, as well as the loss weights corresponding to each loss; calculating the gradient of the hybrid loss function using the backpropagation algorithm to obtain the gradient value; and pruning the gradient value if it exceeds a preset gradient threshold to obtain a pruned gradient value. The model parameters in the student-side model are then updated based on the pruned gradient value to train the student-side model and obtain the target student-side model. The model parameters in the student-side model include embedding layer parameters, encoding layer parameters, and classification layer parameters.

[0187] For example, the calculation process of task classification loss can be represented by the following functional expression: In the formula, This represents the task classification loss; K represents the total number of trust level labels, which is 5 in this example. This indicates that the predicted grading result is the first one. The probability value of each trust level label; Indicates the first in the label The probability value of each trust level label can be understood as the actual trust level label (High Level 1 / High Level 2 / Intermediate Level 1 / Intermediate Level 2 / Low) calculated by the "Expert Scoring Experience Table".

[0188] For example, the hybrid loss function can be expressed as a functional expression as follows: In the formula, Represents the mixed loss function; Indicates task-level loss; This indicates the loss from distillation by multiple teachers; This represents the global feature distillation loss; Indicates regular loss; This represents the loss weight corresponding to the task-level loss; This represents the loss weight corresponding to the multi-teacher distillation loss; This represents the loss weight corresponding to the global feature distillation loss; This represents the loss weight corresponding to the regularization loss.

[0189] In this example, the loss weights corresponding to each loss can be set according to actual needs, for example... , , , .

[0190] For example, the gradient of the mixed loss function is calculated using the backpropagation algorithm to obtain the gradient value, i.e. In the formula, Indicates the gradient value; Represents the mixed loss function; This represents the model parameters in the student-side model.

[0191] For example, a gradient clipping rule can be set after backpropagation and before model parameter updates to avoid gradient explosion caused by excessively large gradient values.

[0192] In this example, the gradient determination rule can be represented by a function expression as follows.

[0193] In the formula, This represents the gradient value after clipping; Indicates the gradient value; The normal form representing the gradient value; This indicates the clipping threshold.

[0194] For example, the student-side model is trained by using an optimizer to update the model parameters based on the pruned gradient values. The optimizer could be, for example, Adaptive Moment Estimation (Adam) or Stochastic Gradient Descent (SGD).

[0195] For example, the batch size and learning rate during the training process of the student-side model can be set according to actual needs. For instance, the batch size can be set to 32 and the learning rate to 0.00002.

[0196] According to the above implementation method, during the training process of the student-side model, firstly, the student-side model is used to predict the original electronic data to obtain the corresponding predicted classification results, thereby enabling preliminary classification and identification of the data. Subsequently, the difference between the predicted classification results and the original electronic data labels is calculated using the cross-entropy loss function to obtain the task classification loss, which helps the student-side model to more accurately fit the true distribution of the data. On this basis, the task classification loss, multi-teacher distillation loss, global feature distillation loss, and regularization loss are combined and assigned their respective loss weights to construct a hybrid loss function, thereby simultaneously considering task performance, teacher knowledge transfer, global feature consistency, and expert prior constraints during training. Next, the gradient of the hybrid loss function is calculated using the backpropagation algorithm to achieve effective transmission of gradient information and support the optimization of model parameters. To avoid excessively large gradients leading to training instability, gradient values ​​are also pruned, and restrictions are imposed when the gradient exceeds a preset threshold, which maintains training stability. Finally, the parameters of the embedding layer, encoding layer, and classification layer of the student-side model are updated based on the pruned gradients to optimize the student-side model parameters, thereby improving the overall performance of the student-side model in data prediction and classification tasks.

[0197] Figure 2 This is a structural block diagram of a graded identification device for electronic data trust strength authentication according to an embodiment of the present invention.

[0198] like Figure 2 As shown, the graded identification device for electronic data trust strength authentication may include:

[0199] The processing module 510 is used to process the original electronic data and the preset prompt word classification template to obtain the sample index sequence;

[0200] The semantic feature extraction module 520 is used to perform deep semantic feature extraction on the sample index sequence through multiple teacher-side models to obtain multiple first feature vectors; and to perform shallow semantic feature extraction on the sample index sequence through a student-side model to obtain second feature vectors.

[0201] The integration module 530 is used to integrate the second feature vector and each of the first feature vectors based on the first teacher weights in each of the teacher-side models to obtain the multi-teacher distillation loss;

[0202] The global feature distillation loss determination module 540 is used to determine the global feature distillation loss based on the squared Euclidean distance between each of the first feature vectors and the second feature vectors.

[0203] The regularization processing module 550 is used to perform regularization processing on each of the first feature vectors through normalization coefficients to obtain regularization loss;

[0204] The hierarchical identification module 560 is used to construct a hybrid loss function based on task hierarchical loss, the multi-teacher distillation loss, the global feature distillation loss, and the regularization loss, train the student-end model through the gradient of the hybrid loss function to obtain the target student-end model, and perform trust strength hierarchical identification on the electronic data to be identified through the target student-end model.

[0205] In one embodiment, the processing module includes:

[0206] The splicing unit is used to splice the original electronic data and the preset prompt word hierarchical template to obtain spliced ​​text;

[0207] The word segmentation unit is used to segment the concatenated text into multiple sub-words;

[0208] The first mapping unit is used to map each of the sub-words using a preset vocabulary to obtain the integer index corresponding to each of the sub-words;

[0209] The sample index sequence determination unit is used to determine the sample index sequence based on the integer index corresponding to each of the sub-words.

[0210] In one embodiment, the semantic feature extraction module includes:

[0211] The second mapping unit is used to map each sample index in the sample index sequence based on the embedding layer parameters in the teacher-side model to obtain the first word embedding matrix;

[0212] The first encoding unit is used to encode the first word embedding matrix based on the encoding layer parameters in the teacher-side model to obtain deep semantic sequence features;

[0213] The first mean pooling unit is used to perform mean pooling on the deep semantic sequence features to obtain the first feature vector;

[0214] The third mapping unit is used to map each sample index in the sample index sequence based on the embedding layer parameters in the student-side model to obtain the second word embedding matrix;

[0215] The second encoding unit is used to encode the second word embedding matrix based on the encoding layer parameters in the student-side model to obtain shallow semantic sequence features;

[0216] The second mean pooling unit is used to perform mean pooling on the shallow semantic sequence features to obtain the second feature vector.

[0217] In one embodiment, the integration module includes:

[0218] A softening processing unit is used to soften the second feature vector and temperature coefficient through an activation function to obtain a first probability distribution;

[0219] The loss calculation unit is configured to perform the following loss calculation for each of the first feature vectors:

[0220] A softening subunit is used to soften the first feature vector and the temperature coefficient through the activation function to obtain a second probability distribution;

[0221] The first relative entropy calculation subunit is used to calculate the relative entropy of the first probability distribution and the second probability distribution to obtain the first divergence value;

[0222] The second relative entropy calculation subunit is used to determine the third probability distribution based on the mean of the first probability distribution and the second probability distribution; and to calculate the relative entropy between the first probability distribution and the second probability distribution and the third probability distribution to obtain the second divergence value and the third divergence value.

[0223] The fourth divergence value determination sub-unit is used to determine the fourth divergence value based on the mean of the second and third divergence values;

[0224] The first weighted summation subunit is used to perform a weighted summation of the first divergence value and the fourth divergence value based on the weight coefficients to obtain the combined loss term;

[0225] The second weighted summation subunit is used to determine each first calculation result based on the product of each combined loss term and the temperature square factor when the loss calculation is performed on each of the first feature vectors; and to perform weighted summation on each of the first calculation results based on the first teacher weight in each of the teacher-side models to obtain the multi-teacher distillation loss.

[0226] In one embodiment, the global feature distillation loss determination module includes:

[0227] The first feature distillation loss unit is used to determine the first feature distillation loss for each of the first feature vectors based on the squared Euclidean distance between the first feature vector and the second feature vector.

[0228] The weighted summation unit is used to perform weighted summation on each of the first feature distillation losses based on the second teacher weights in the multiple teacher-side models, to obtain the global feature distillation loss.

[0229] In one implementation, the regularization processing module includes:

[0230] The combination unit is used to combine teachers in multiple teacher-side models to obtain multiple teacher pairs;

[0231] The second feature distillation loss unit is used to determine the second feature distillation loss of a teacher pair based on the squared Euclidean distance between the first feature vector corresponding to the first teacher in the teacher pair and the first feature vector corresponding to the second teacher in the teacher pair.

[0232] A regularization loss unit is used to determine the regularization loss based on the sum of the second feature distillation losses of each teacher pair and the product of the normalization coefficient.

[0233] In one embodiment, the hierarchical identification module includes:

[0234] The first prediction unit is used to predict the original electronic data through the student-side model to obtain the prediction grading result corresponding to the original electronic data.

[0235] The cross-entropy calculation unit is used to calculate the cross-entropy between the predicted classification result corresponding to the original electronic data and the label corresponding to the original electronic data through the cross-entropy loss function, so as to obtain the task classification loss.

[0236] The hybrid loss function determination unit is used to determine the hybrid loss function based on the task-level loss, the multi-teacher distillation loss, the global feature distillation loss, and the regularization loss, as well as the loss weights corresponding to each loss.

[0237] The gradient calculation unit is used to calculate the gradient of the hybrid loss function using the backpropagation algorithm to obtain the gradient value;

[0238] The student-side model training unit is used to clip the gradient value when the gradient value is greater than a preset gradient threshold, obtain the clipped gradient value, and update the model parameters in the student-side model based on the clipped gradient value to train the student-side model. The model parameters in the student-side model include embedding layer parameters, encoding layer parameters, and classification layer parameters.

[0239] The specific functions and examples of each module and submodule of the system in this embodiment of the invention can be found in the relevant descriptions of the corresponding steps in the above method embodiments, and will not be repeated here.

[0240] The acquisition, storage, and application of user personal information involved in the technical solution of this invention all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0241] This invention also provides a graded identification system for electronic data trust strength authentication, comprising:

[0242] At least one processor; and a memory communicatively connected to said at least one processor;

[0243] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method described in any one of the embodiments of the present invention.

[0244] The beneficial effects of the hierarchical identification system for electronic data trust strength authentication in this embodiment of the invention are equivalent to the beneficial effects of the hierarchical identification method for electronic data trust strength authentication described above, and will not be repeated here.

[0245] This invention also provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to perform the method described in any one of the embodiments of this invention.

[0246] The beneficial effects of the storage medium of the present invention are equivalent to the beneficial effects of the graded identification method for electronic data trust strength authentication described above, and will not be repeated here.

[0247] Figure 3 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present invention is shown. Electronic device 800 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic device 800 may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0248] like Figure 3 As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. The RAM 803 may also store various programs and data required for the operation of the electronic device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0249] Multiple components in electronic device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of displays, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows electronic device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0250] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the hierarchical identification method for electronic data trust strength authentication. For example, in some embodiments, the hierarchical identification method for electronic data trust strength authentication can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the hierarchical identification method for electronic data trust strength authentication described above can be performed. Alternatively, in other embodiments, the computing unit 801 may be configured by any other suitable means (e.g., by means of firmware) to perform a graded identification method for electronic data trust strength authentication.

[0251] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0252] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0253] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0254] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0255] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0256] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0257] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0258] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this invention should be included within the scope of protection of this invention.< / unk>

Claims

1. A method for hierarchical identification of electronic data trust strength authentication, characterized in that, The method comprises the following steps: processing original electronic data and a preset prompt word grading template to obtain a sample index sequence; extracting deep semantic features of the sample index sequence through multiple teacher models to obtain multiple first feature vectors; and extracting shallow semantic features of the sample index sequence through a student model to obtain a second feature vector; integrating the second feature vector and each of the first feature vectors based on a first teacher weight in each of the teacher models to obtain a multi-teacher distillation loss; determining a global feature distillation loss based on the squared Euclidean distance between each of the first feature vectors and the second feature vector; regularizing each of the first feature vectors through a normalization coefficient to obtain a regularization loss; constructing a hybrid loss function based on a task grading loss, the multi-teacher distillation loss, the global feature distillation loss, and the regularization loss, training the student model through the gradient of the hybrid loss function to obtain a target student model, and performing trust strength grading identification on to-be-identified electronic data through the target student model.

2. The method of claim 1, wherein, The processing of the original electronic data and the preset prompt word grading template to obtain the sample index sequence comprises: splicing the original electronic data and the preset prompt word grading template to obtain a spliced text; performing word segmentation on the spliced text to obtain multiple subwords; mapping each of the subwords through a preset word table to obtain an integer index corresponding to each of the subwords; determining the sample index sequence based on the integer index corresponding to each of the subwords.

3. The method of claim 1, wherein, The deep semantic feature extraction of the sample index sequence through multiple teacher models to obtain multiple first feature vectors comprises: mapping each sample index in the sample index sequence based on the embedding layer parameters in the teacher model to obtain a first word embedding matrix; encoding the first word embedding matrix based on the encoding layer parameters in the teacher model to obtain deep semantic sequence features; performing mean pooling on the deep semantic sequence features to obtain the first feature vectors; mapping each sample index in the sample index sequence based on the embedding layer parameters in the student model to obtain a second word embedding matrix; encoding the second word embedding matrix based on the encoding layer parameters in the student model to obtain shallow semantic sequence features; performing mean pooling on the shallow semantic sequence features to obtain the second feature vector. The integration of the second feature vector and each of the first feature vectors based on the first teacher weight in each of the teacher models to obtain a multi-teacher distillation loss comprises: softening the second feature vector and a temperature coefficient through an activation function to obtain a first probability distribution; 4. The method of claim 1, wherein, for each of the first feature vectors, the following loss calculation is performed respectively: ​ ​ The first feature vector and the temperature coefficient are softened by the activation function to obtain a second probability distribution; The first probability distribution and the second probability distribution are calculated by relative entropy to obtain a first divergence value; Based on the mean of the first probability distribution and the second probability distribution, a third probability distribution is determined; the first probability distribution and the second probability distribution are respectively calculated by relative entropy with the third probability distribution to obtain a second divergence value and a third divergence value; Based on the mean of the second divergence value and the third divergence value, a fourth divergence value is determined; The first divergence value and the fourth divergence value are weighted and summed based on the weight coefficient to obtain a combined loss term; In the case of performing the loss calculation on each of the first feature vectors, based on the product of each of the combined loss terms and a temperature square factor, a first calculation result is determined; based on the first teacher weight in each of the teacher end models, each of the first calculation results is weighted and summed to obtain the multi-teacher distillation loss.

5. The method of claim 1, wherein, The global feature distillation loss is determined based on the squared Euclidean distance between each of the first feature vectors and the second feature vectors, including: For each of the first feature vectors, the first feature distillation loss is determined based on the squared Euclidean distance between the first feature vector and the second feature vector; The global feature distillation loss is obtained by weighting and summing each of the first feature distillation losses based on the second teacher weight in the plurality of teacher end models.

6. The method of claim 1, wherein, The regularization loss is obtained by normalizing each of the first feature vectors, including: The teachers in the plurality of teacher end models are combined to obtain a plurality of teacher pairs; For a plurality of teacher pairs, the second feature distillation loss of the teacher pair is determined based on the squared Euclidean distance between the first feature vector corresponding to the first teacher in the teacher pair and the first feature vector corresponding to the second teacher in the teacher pair; The regularization loss is determined based on the product of the sum of the second feature distillation loss of each of the teacher pairs and the normalization coefficient.

7. The method of claim 1, wherein, The mixed loss function is constructed based on the task grading loss, the multi-teacher distillation loss, the global feature distillation loss and the regularization loss, and the target student end model is obtained by training the student end model through the gradient of the mixed loss function, including: The original electronic data is predicted by the student end model to obtain a predicted grading result corresponding to the original electronic data; The cross-entropy between the predicted grading result corresponding to the original electronic data and the label corresponding to the original electronic data is calculated by the cross-entropy loss function to obtain the task grading loss; The mixed loss function is determined based on the task grading loss, the multi-teacher distillation loss, the global feature distillation loss and the regularization loss, and the loss weight corresponding to each loss; The gradient value is obtained by calculating the gradient of the mixed loss function through the back propagation algorithm; In a case where the gradient value is greater than a preset gradient threshold value, the gradient value is clipped to obtain a clipped gradient value, and a model parameter in the student end model is updated based on the clipped gradient value to train the student end model to obtain the target student end model, wherein the model parameter in the student end model includes an embedding layer parameter, an encoding layer parameter and a classification layer parameter.

8. An electronic data trust strength authentication hierarchical identification apparatus, characterized by, Comprise: The processing module is used for processing the original electronic data and the preset prompt word grading template to obtain a sample index sequence; The semantic feature extraction module is used for deep layer semantic feature extraction on the sample index sequence through a plurality of teacher end models to obtain a plurality of first feature vectors; And The sample index sequence is extracted through a shallow layer semantic feature extraction of the student end model to obtain a second feature vector; The integration module is used for integrating the second feature vector and each first feature vector based on a first teacher weight in each teacher end model to obtain a multi-teacher distillation loss; The global feature distillation loss determination module is used for determining a global feature distillation loss based on the Euclidean distance square of each first feature vector and the second feature vector; The regularization processing module is used for normalizing each first feature vector through a normalization coefficient to obtain a regularization loss; The hierarchical identification module is used for constructing a hybrid loss function based on a task hierarchical loss, the multi-teacher distillation loss, the global feature distillation loss and the regularization loss, training the student end model through the gradient of the hybrid loss function to obtain a target student end model, and identifying the trust strength of the to-be-identified electronic data through the target student end model.

9. The apparatus of claim 8, wherein, The processing module comprises: The splicing unit is used for splicing the original electronic data and the preset prompt word grading template to obtain a spliced text; The word segmentation unit is used for word segmentation on the spliced text to obtain a plurality of subwords; The first mapping unit is used for mapping each subword through a preset word table to obtain an integer index corresponding to each subword; The sample index sequence determination unit is used for determining the sample index sequence based on the integer index corresponding to each subword.

10. The apparatus of claim 8, wherein, The semantic feature extraction module comprises: The second mapping unit is used for mapping each sample index in the sample index sequence based on the embedding layer parameter in the teacher end model to obtain a first word embedding matrix; The first encoding unit is used for encoding the first word embedding matrix based on the encoding layer parameter in the teacher end model to obtain a deep layer semantic sequence feature; The first mean pooling unit is used for mean pooling on the deep layer semantic sequence feature to obtain the first feature vector; The third mapping unit is used for mapping each sample index in the sample index sequence based on the embedding layer parameter in the student end model to obtain a second word embedding matrix; The second encoding unit is used for encoding the second word embedding matrix based on the encoding layer parameter in the student end model to obtain a shallow layer semantic sequence feature; A second mean pooling unit is configured to perform mean pooling on the shallow semantic sequence feature to obtain the second feature vector.

11. The apparatus of claim 8, wherein, The integration module comprises: A softening processing unit is configured to perform softening processing on the second feature vector and a temperature coefficient by using an activation function to obtain a first probability distribution. The loss calculation unit is configured to perform the following loss calculation for each of the first feature vectors respectively: A softening processing subunit is configured to perform softening processing on the first feature vector and the temperature coefficient by using the activation function to obtain a second probability distribution. A first relative entropy calculation subunit is configured to perform relative entropy calculation on the first probability distribution and the second probability distribution to obtain a first divergence value. A second relative entropy calculation subunit is configured to determine a third probability distribution based on the mean of the first probability distribution and the second probability distribution, and perform relative entropy calculation on the first probability distribution and the second probability distribution respectively with the third probability distribution to obtain a second divergence value and a third divergence value. A fourth divergence value determination subunit is configured to determine a fourth divergence value based on the mean of the second divergence value and the third divergence value. A first weighted sum subunit is configured to perform weighted sum on the first divergence value and the fourth divergence value based on a weight coefficient to obtain a combined loss term. A second weighted sum subunit is configured to determine a first calculation result based on the product of each combined loss term and a temperature square factor in the case that the loss calculation is performed on each of the first feature vectors, and perform weighted sum on each first calculation result based on a first teacher weight in each of the teacher end models to obtain the multi-teacher distillation loss.

12. The apparatus of claim 8, wherein, The global feature distillation loss determination module comprises: A first feature distillation loss unit is configured to determine a first feature distillation loss based on the squared Euclidean distance between the first feature vector and the second feature vector for each of the first feature vectors. A weighted sum unit is configured to perform weighted sum on each first feature distillation loss based on a second teacher weight in the plurality of teacher end models to obtain the global feature distillation loss.

13. The apparatus of claim 8, wherein, The regularization processing module comprises: A combination unit is configured to combine teachers in the plurality of teacher end models to obtain a plurality of teacher pairs. A second feature distillation loss unit is configured to determine a second feature distillation loss of a teacher pair based on the squared Euclidean distance between the first feature vector corresponding to a first teacher in the teacher pair and the first feature vector corresponding to a second teacher in the teacher pair for the teacher pair. A regularization loss unit is configured to determine the regularization loss based on the product of the sum of the second feature distillation losses of each teacher pair and the normalization coefficient.

14. The apparatus of claim 8, wherein, The hierarchical identification module comprises: A first prediction unit is configured to predict the original electronic data by using the student end model to obtain a predicted hierarchical result corresponding to the original electronic data. The cross-entropy calculation unit is configured to calculate cross-entropy between a predicted classification result corresponding to the original electronic data and a label corresponding to the original electronic data by using a cross-entropy loss function, and obtain the task classification loss. The mixed loss function determination unit is configured to determine the mixed loss function based on the task classification loss, the multi-teacher distillation loss, the global feature distillation loss, the regularization loss, and loss weights corresponding to the losses. The gradient calculation unit is configured to calculate a gradient of the mixed loss function by using a back propagation algorithm, and obtain a gradient value. The student-end model training unit is configured to clip the gradient value when the gradient value is greater than a preset gradient threshold, obtain a clipped gradient value, and update model parameters in the student-end model based on the clipped gradient value, so as to train the student-end model. The model parameters in the student-end model include embedding layer parameters, encoding layer parameters, and classification layer parameters.

15. An electronic data trust strength authentication hierarchical identification system characterized by, comprise: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

16. A non-transitory computer-readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to perform the method of any one of claims 1-7.

Images