Feature label filtering method, apparatus, and system, electronic device, medium, and product

Abstract

Embodiments of the present application provide a feature label filtering method, apparatus, and system, an electronic device, a medium, and a product. The method comprises: acquiring a plurality of original training samples; for each original training sample, after the original training sample is input into an attention model, obtaining a key matrix and a value matrix, and, on the basis of the key matrix and the value matrix, determining an attention loss corresponding to the original training sample; on the basis of the attention losses respectively corresponding to the original training samples, determining a first total attention loss; on the basis of a plurality of target training samples and an attention model, determining a second total attention loss, wherein the target training samples comprise at least one first feature label, and the first feature label is filtered from a plurality of original feature labels; and on the basis of the first total attention loss and the second total attention loss, determining whether to delete a second feature label, the second feature label being a feature label other than the at least one first feature label. Thus, the accuracy of feature label filtering can be effectively improved.

Description

Feature label screening method, device, system, electronic equipment, medium and product

[0001] The present application claims priority to the Chinese patent application No. 202411391275.2, filed on September 30, 2024, and entitled "Feature label screening method, device, system, electronic equipment, medium and product", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the field of artificial intelligence, and in particular to a feature label screening method, device, system, electronic equipment, medium and product. BACKGROUND

[0003] With the continuous iteration and innovation of financial products and financial businesses, the number and types of labels are increasing. Although the increase in the number of labels effectively expands user portrait features and the overall label library, redundant labels increase noise in the data, making it more difficult to extract valuable information therefrom, and may also cause data confusion, affecting the accuracy and reliability of analysis results.

[0004] Therefore, how to effectively screen labels of real value needs to be solved. SUMMARY

[0005] The present application provides a feature label screening method, device, system, electronic equipment, medium and product to improve the accuracy of feature label screening.

[0006] In a first aspect, the present application provides a feature label screening method, comprising:

[0007] Obtaining a plurality of original training samples, each original training sample comprising a plurality of original feature labels and feature values corresponding to each original feature label;

[0008] For each original training sample, after the original training sample is input into an attention model, a key matrix and a value matrix are obtained, and an attention loss corresponding to the original training sample is determined according to the key matrix and the value matrix, the attention model comprising an encoder and a decoder, the key matrix and the value matrix being passed from the encoder to the decoder;

[0009] Determining a first total attention loss according to the attention loss corresponding to each original training sample;

[0010] determine a second total attention loss based on the plurality of target training samples and the attention model, wherein the target training samples comprise at least one first feature label and a feature value corresponding to each first feature label, and the first feature label is selected from a plurality of original feature labels comprised in the original training sample;

[0011] determine whether to delete a second feature label based on the first total attention loss and the second total attention loss, wherein the second feature label is a feature label other than the at least one first feature label in the plurality of original feature labels.

[0012] In a possible implementation, the attention loss comprises a first attention loss; and determining the attention loss corresponding to the original training sample based on the key matrix and the value matrix comprises:

[0013] obtaining a real key matrix and a real value matrix corresponding to the original training sample;

[0014] determining a first key-related loss based on the key matrix and the real key matrix, and determining a first value-related loss based on the value matrix and the real value matrix;

[0015] determining the first attention loss corresponding to the original training sample based on the first key-related loss and the first value-related loss.

[0016] In a possible implementation, determining the first key-related loss based on the key matrix and the real key matrix, and determining the first value-related loss based on the value matrix and the real value matrix comprises:

[0017] calculating a first mutual information of the key matrix and the real key matrix, and determining the first key-related loss based on the first mutual information;

[0018] calculating a second mutual information of the value matrix and the real value matrix, and determining the first value-related loss based on the second mutual information.

[0019] In a possible implementation, determining the first key-related loss based on the first mutual information comprises:

[0020] calculating a first Pearson correlation coefficient of the key matrix and the real key matrix;

[0021] adding the first mutual information and the first Pearson correlation coefficient to obtain a first sum;

[0022] determining half of the first sum as the first key-related loss;

[0023] Correspondingly, determining the first value-related loss based on the second mutual information comprises:

[0024] calculating a second Pearson correlation coefficient of the value matrix and the ground truth matrix;

[0025] adding the second mutual information and the second Pearson correlation coefficient to obtain a second sum;

[0026] determining half of the second sum as the first value correlation loss.

[0027] In a possible implementation, the attention loss includes a second attention loss; after the original training sample is input into an attention model, a key matrix and a value matrix are obtained, and the second attention loss corresponding to the original training sample is determined according to the key matrix and the value matrix, including:

[0028] after the original training sample is input into an attention model, a key matrix, a value matrix and a query matrix are obtained;

[0029] determining the second attention loss corresponding to the original training sample according to the key matrix, the value matrix and the query matrix.

[0030] In a possible implementation, the second attention loss corresponding to the original training sample is determined according to the key matrix, the value matrix and the query matrix, including:

[0031] obtaining a ground truth key matrix, a ground truth value matrix and a ground truth query matrix corresponding to the original training sample;

[0032] determining a second key correlation loss according to the key matrix and the ground truth key matrix;

[0033] determining a second value correlation loss according to the value matrix and the ground truth value matrix;

[0034] determining a query correlation loss according to the query matrix and the ground truth query matrix;

[0035] determining the second attention loss corresponding to the original training sample according to the second key correlation loss, the second value correlation loss and the query correlation loss.

[0036] In a possible implementation, the ground truth key matrix, the ground truth value matrix and the ground truth query matrix each include a plurality of elements, for each element, if an element value corresponding to the element is a first preset value, it indicates that a position where the element is located should be paid attention to, and if the element value corresponding to the element is a second preset value, it indicates that the position where the element is located should not be paid attention to.

[0037] In a possible implementation, the second key correlation loss is determined according to the key matrix and the ground truth key matrix, including:

[0038] scaling element values corresponding to elements in the key matrix to between a first preset value and a second preset value to obtain a scaled key matrix;

[0039] determining a second key-related loss according to the scaled key matrix and the real key matrix.

[0040] In a possible implementation, determining the second key-related loss according to the scaled key matrix and the real key matrix includes:

[0041] calculating a difference value between an element value corresponding to each element in the scaled key matrix and an element value corresponding to a target element, wherein a relative position of the target element in the real key matrix is the same as a relative position of the element in the key matrix;

[0042] calculating a sum of the difference values corresponding to the elements in the scaled key matrix respectively to obtain a third sum;

[0043] determining the second key-related loss according to the third sum.

[0044] In a possible implementation, determining the second key-related loss according to the third sum includes:

[0045] calculating an intersection over union of the key matrix and the real key matrix,

[0046] determining a product of the intersection over union and the third sum as the second key-related loss.

[0047] In a possible implementation, after the original training sample is input into the attention model, a key matrix and a value matrix are obtained, and determining an attention loss corresponding to the original training sample according to the key matrix and the value matrix includes:

[0048] after the original training sample is input into the attention model, obtaining a key matrix, a value matrix and a query matrix;

[0049] determining a first attention loss based on the key matrix and the value matrix;

[0050] determining a second attention loss based on the key matrix, the value matrix and the query matrix;

[0051] determining the attention loss corresponding to the original training sample based on the first attention loss and the second attention loss.

[0052] In a possible implementation, the method further includes:

[0053] obtaining a sample label corresponding to each original training sample;

[0054] For each original training sample, determine an output corresponding to the original training sample, and determine an output loss corresponding to the original training sample according to a sample label and the output corresponding to the original training sample;

[0055] Determine a first total output loss according to the output loss corresponding to each original training sample;

[0056] For each target training sample, determine an output corresponding to the target training sample, and determine an output loss corresponding to the target training sample according to a sample label and the output corresponding to the target training sample;

[0057] Determine a second total output loss according to the output loss corresponding to each target training sample;

[0058] Accordingly, determine whether to delete the second feature label according to the first total attention loss and the second total attention loss, including:

[0059] Determine whether to delete the second feature label according to the first total attention loss, the second total attention loss, the first total output loss, and the second total output loss.

[0060] In a possible implementation, determining whether to delete the second feature label according to the first total attention loss, the second total attention loss, the first total output loss, and the second total output loss includes:

[0061] Determine a first total loss according to the first total attention loss and the first total output loss;

[0062] Determine a second total loss according to the second total attention loss and the second total output loss;

[0063] When the difference between the second total loss and the first total loss is less than a preset threshold, determine to delete the second feature label.

[0064] In a possible implementation, determining the first total loss according to the first total attention loss and the first total output loss includes:

[0065] Obtain a first weight corresponding to the first total attention loss and a second weight corresponding to the first total output loss;

[0066] Determine the first total loss according to the first total attention loss, the first total output loss, the first weight, and the second weight.

[0067] In a possible implementation, the output corresponding to each original training sample is a classification probability value, the classification probability value is used to indicate a probability that the original training sample is classified into each of a plurality of preset categories, and the output loss corresponding to the original training sample is determined according to the sample label and the output corresponding to the original training sample, including:

[0068] The output loss corresponding to the original training sample is determined according to the following formula:

[0069] wherein l represents the output loss, M represents a total number of the plurality of preset categories, C represents a category, P c represents the probability of being classified into the Cth category, y c represents the sample label, and γ is a constant greater than 1.

[0070] In a possible implementation, the output corresponding to each original training sample is a classification probability value, the classification probability value is used to indicate a probability that the original training sample is classified into each of a plurality of preset categories, and the output loss corresponding to the original training sample is determined according to the sample label and the output corresponding to the original training sample, including:

[0071] The output loss corresponding to the original training sample is determined according to the following formula:

[0072] wherein l represents the output loss, M represents a total number of the plurality of preset categories, C represents a category, P c represents the probability of being classified into the Cth category, y c represents the sample label, and γ is a constant greater than 1.

[0073] In a second aspect, an embodiment of the present application provides a feature label screening method, applied to a client, including:

[0074] Obtaining a plurality of local training samples, each local training sample including a plurality of local feature labels and feature values corresponding to each local feature label;

[0075] For each local training sample, the feature values of each local feature label corresponding to the local training sample are encrypted to obtain an original training sample, wherein the encryption manner is to add Laplace noise;

[0076] sending a plurality of original training samples to a server, each original training sample including a plurality of original feature labels and feature values corresponding to the original feature labels; so that the server, for each original training sample, obtains a key matrix and a value matrix after the original training sample is input into an attention model, and determines an attention loss corresponding to the original training sample according to the key matrix and the value matrix, the attention model including an encoder and a decoder, and the key matrix and the value matrix being transmitted by the encoder to the decoder; determining a first total attention loss according to the attention loss corresponding to each original training sample respectively; determining a second total attention loss based on a plurality of target training samples and the attention model, the target training samples including at least one first feature label and feature values corresponding to the first feature label, the first feature label being selected from the plurality of original feature labels included in the original training sample; and determining whether to delete a second feature label according to the first total attention loss and the second total attention loss, the second feature label being a feature label other than the at least one first feature label in the plurality of original feature labels.

[0077] In a possible implementation, the method further includes:

[0078] normalizing the feature values corresponding to each local feature label to a preset dimension;

[0079] correspondingly, encrypting the feature values of each local feature label corresponding to the local training sample to obtain an original training sample, including:

[0080] encrypting each normalized feature value corresponding to the local training sample to obtain an original training sample.

[0081] In a third aspect, an embodiment of the present application provides a feature label screening system, including:

[0082] a server configured to execute the method of the first aspect;

[0083] a client configured to execute the method of the second aspect.

[0084] In a fourth aspect, an embodiment of the present application provides a feature label screening device, including:

[0085] an acquisition module configured to acquire a plurality of original training samples, each original training sample including a plurality of original feature labels and feature values corresponding to the original feature labels;

[0086] The first determining module is configured to, for each original training sample, obtain a key matrix and a value matrix after the original training sample is input into an attention model, and determine an attention loss corresponding to the original training sample according to the key matrix and the value matrix, wherein the attention model comprises an encoder and a decoder, and the key matrix and the value matrix are transmitted by the encoder to the decoder;

[0087] The second determining module is configured to determine a first total attention loss according to the attention losses respectively corresponding to the original training samples.

[0088] The third determining module is configured to determine a second total attention loss based on the target training samples and the attention model, wherein the target training samples comprise at least one first feature label and feature values corresponding to each first feature label, and the first feature label is selected from a plurality of original feature labels included in the original training samples.

[0089] The fourth determining module is configured to determine whether to delete a second feature label according to the first total attention loss and the second total attention loss, wherein the second feature label is a feature label other than the at least one first feature label in the plurality of original feature labels.

[0090] In a fourth aspect, an embodiment of the present application provides a feature label screening device applied to a client, comprising:

[0091] The obtaining module is configured to obtain a plurality of local training samples, and each local training sample comprises a plurality of local feature labels and feature values corresponding to each local feature label.

[0092] The encryption module is configured to, for each local training sample, encrypt the feature values of each local feature label corresponding to the local training sample to obtain an original training sample, wherein the encryption manner is to add Laplace noise.

[0093] The sending module is configured to send a plurality of original training samples to a server, each original training sample comprising a plurality of original feature labels and feature values corresponding to the original feature labels; so that the server, for each original training sample, obtains a key matrix and a value matrix after the original training sample is input into an attention model, and determines an attention loss corresponding to the original training sample according to the key matrix and the value matrix, the attention model comprising an encoder and a decoder, and the key matrix and the value matrix being transmitted by the encoder to the decoder; determines a first total attention loss according to the attention loss corresponding to each original training sample; determines a second total attention loss based on a plurality of target training samples and the attention model, the target training samples comprising at least one first feature label and feature values corresponding to the first feature label, the first feature label being selected from the plurality of original feature labels comprised in the original training samples; and determines whether to delete a second feature label according to the first total attention loss and the second total attention loss, the second feature label being a feature label other than the at least one first feature label in the plurality of original feature labels.

[0094] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising: a memory, a processor;

[0095] The memory stores computer-executable instructions.

[0096] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the method in any of the preceding aspects.

[0097] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method in any of the preceding aspects.

[0098] In a seventh aspect, an embodiment of the present application provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the method in any of the preceding aspects.

[0099] The feature label screening method, device, system, electronic equipment, medium and product provided by the embodiments of the present application, by obtaining a plurality of original training samples, each original training sample comprising a plurality of original feature labels and feature values corresponding to each original feature label; for each original training sample, after the original training sample is input into an attention model, a key matrix and a value matrix are obtained, and the attention loss corresponding to the original training sample is determined according to the key matrix and the value matrix, the attention model comprising an encoder and a decoder, and the key matrix and the value matrix being transmitted by the encoder to the decoder; the first total attention loss is determined according to the attention loss corresponding to each original training sample respectively; the second total attention loss is determined based on a plurality of target training samples and the attention model; the target training sample comprises at least one first feature label and feature values corresponding to each first feature label; the first feature label is screened from the plurality of original feature labels comprised in the original training sample; and whether to delete a second feature label is determined according to the first total attention loss and the second total attention loss, wherein the second feature label is a feature label other than the at least one first feature label in the plurality of original feature labels. The present application creatively takes the attention loss as an index for measuring the importance of the feature label, which can effectively improve the accuracy of feature label screening, and by screening the most important feature label, the interference of noise and redundant information can be reduced, the model can be more focused on the key features, and the generalization ability and prediction accuracy of the model can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0100] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0101] FIG. 1 is an application scenario diagram provided by an embodiment of the present application;

[0102] FIG. 2 is a flow diagram of a feature label screening method provided by an embodiment of the present application;

[0103] FIG. 3 is a diagram of a real key matrix provided by an embodiment of the present application;

[0104] FIG. 4 is a diagram of the relationship between the output loss and the classification probability value provided by an embodiment of the present application;

[0105] FIG. 5 is another feature label screening method provided by an embodiment of the present application;

[0106] FIG. 6 is a diagram of customer group feature label clustering and dimensionality reduction provided by an embodiment of the present application;

[0107] FIG. 7 is a real attention matrix provided by an embodiment of the present application;

[0108] FIG. 8 is a diagram illustrating a difference calculation according to an embodiment of the present application;

[0109] FIG. 9 is a diagram illustrating image attention drift and IOU according to an embodiment of the present application;

[0110] FIG. 10 is a diagram illustrating a structure of a feature label screening device according to an embodiment of the present application;

[0111] FIG. 11 is a diagram illustrating a structure of another feature label screening device according to an embodiment of the present application;

[0112] FIG. 12 is a diagram illustrating a structure of an electronic device according to an embodiment of the present application.

[0113] The above-described embodiments have been shown and described, and more detailed descriptions will be given hereinafter. The drawings and the written description are not intended to restrict the scope of the present application by any means, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0114] The exemplary embodiments will be described in detail with reference to the accompanying drawings. The following description is merely exemplary in nature and is not intended to limit the scope of the present application, as described. Conversely, the exemplary embodiments described herein are presented by way of example to illustrate the inventive concept of the present application.

[0115] With the continuous iteration and innovation of financial products and financial services, the number and types of labels are increasing. While the increase in the number of labels effectively expands the user portrait features and the overall label library, redundant labels not only increase the noise in the data, making it more difficult to extract valuable information, but also can lead to data confusion, affecting the accuracy and reliability of the analysis results. This phenomenon is particularly prominent in big data analysis and machine learning models, as redundant labels increase the complexity of the model, leading to overfitting problems, which reduces the generalization ability of the model. In addition, redundant labels also increase the cost of data storage and processing, affecting the performance and efficiency of the system.

[0116] Therefore, the application provides a feature label screening method, which can obtain a plurality of original training samples, each original training sample including a plurality of original feature labels and feature values corresponding to each original feature label; for each original training sample, the original training sample is input into an attention model to obtain a key matrix and a value matrix, and the attention loss corresponding to the original training sample is determined according to the key matrix and the value matrix, the attention model including an encoder and a decoder, and the key matrix and the value matrix being transmitted by the encoder to the decoder; the first total attention loss is determined according to the attention loss corresponding to each original training sample; for each original training sample, at least one first feature label is screened from the plurality of original feature labels corresponding to the original training sample, and the at least one first feature label and the feature values corresponding to each first feature label constitute a target training sample; the plurality of target training samples are input into the attention model respectively to determine the second total attention loss; wherein the method for determining the second total attention loss is the same as the method for determining the first total attention loss, and whether to delete a second feature label is determined according to the first total attention loss and the second total attention loss, wherein the second feature label is used to indicate the feature labels other than the first feature labels in the original feature labels. The application creatively takes the attention loss as an important index for measuring the importance of the feature labels, which can effectively improve the accuracy of feature label screening, and by screening the most important feature labels, the interference of noise and redundant information can be reduced, the model can be more focused on the key features, and the generalization ability and prediction accuracy of the model can be improved.

[0117] FIG. 1 is an application scenario provided by an embodiment of the application, as shown in FIG. 1, a server first obtains a plurality of original training samples, for each original training sample, the original training sample is input into an encoder in an attention model to obtain a key matrix and a value matrix, the attention model including an encoder and a decoder, and the key matrix and the value matrix being transmitted by the encoder to the decoder, and the attention loss corresponding to the original training sample is determined according to the key matrix and the value matrix; the first total attention loss is determined according to the attention loss corresponding to each original training sample; for each original training sample, at least one first feature label is screened from the plurality of original feature labels corresponding to the original training sample, and a target training sample is constituted; for each target training sample, the target training sample is input into the encoder in the attention model to obtain a corresponding key matrix and value matrix, and the attention loss corresponding to the target training sample is determined according to the obtained key matrix and value matrix, the second total attention loss is determined according to the attention loss corresponding to each target training sample, and whether to delete a second feature label is determined according to the first total attention loss and the second total attention loss, wherein the second feature label is used to indicate the feature labels other than the first feature labels in the original feature labels.

[0118] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards, and provide corresponding operation portal for user to choose authorization or refusal.

[0119] The technical solutions of the present application and how the technical solutions of the present application solve the above technical problems will be described in detail below with specific examples. The following specific examples can be combined with each other, and the same or similar concepts or processes may not be described again in some examples. The embodiments of the present application will be described below with reference to the drawings.

[0120] FIG. 2 is a flowchart of a feature label screening method provided by an embodiment of the present application. The execution subject of the present embodiment can be any device with data processing function, and the present application takes a server as the execution subject for specific description. As shown in FIG. 2, the feature label screening method provided by an embodiment of the present application can include:

[0121] Step 201, obtaining a plurality of original training samples, each original training sample including a plurality of original feature labels and feature values corresponding to each original feature label.

[0122] The feature label screening method provided by the present application can be applied in any field with feature label screening needs, which is not limited by the present application.

[0123] For example, when applied in the financial field, the original feature label can be used to indicate the basic information, financial information, credit information, investment behavior, transaction behavior, account information, geographic information, behavior information, etc. of the user. For example, the original feature label of each original training sample can include the age, gender, marital status, total assets, total liabilities, expectation record, default record, transaction amount, transaction frequency, etc. of the user.

[0124] For example, when applied in the image field, the original feature label can be used to indicate the basic features and high-level features of the image, wherein the basic features of the image can include the color value (such as RGB (Red, Green, Blue) value) of each pixel, image size, etc., and the high-level features of the image can be extracted on the basis of the basic features of the image, such as texture features, shape features, semantic features, frequency features, etc. For example, the original feature label of each original training sample can include the color value of each pixel, the image size, the texture features, the shape features, etc.

[0125] Specifically, the server can obtain a plurality of original training samples, each of which includes a plurality of original feature labels and feature values corresponding to the original feature labels.

[0126] In step 202, for each original training sample, after the original training sample is input into an attention model, a key matrix and a value matrix are obtained, and an attention loss corresponding to the original training sample is determined according to the key matrix and the value matrix, the attention model including an encoder and a decoder, and the key matrix and the value matrix being transmitted by the encoder to the decoder.

[0127] The key matrix is obtained by applying a linear transformation to an input sequence corresponding to the original training sample, and can represent the features of each element in the input sequence.

[0128] The value matrix is obtained by applying another linear transformation to the input sequence corresponding to the original training sample, and can represent the actual content of each element in the input sequence.

[0129] Specifically, the attention model includes an encoder and a decoder, and for each original training sample, the server inputs the original training sample into the encoder in the attention model, outputs a key matrix and a value matrix in the encoder, and inputs the output key matrix and the value matrix into the decoder. The server determines the corresponding attention loss of the original training sample according to the key matrix and the value matrix transmitted by the encoder to the decoder.

[0130] Optionally, the attention loss includes a first attention loss, and determining the attention loss corresponding to the original training sample according to the key matrix and the value matrix includes:

[0131] obtaining a real key matrix and a real value matrix corresponding to the original training sample;

[0132] determining a first key-related loss according to the key matrix and the real key matrix, and determining a first value-related loss according to the value matrix and the real value matrix;

[0133] determining the first attention loss corresponding to the original training sample according to the first key-related loss and the first value-related loss.

[0134] The determination methods of the real key matrix and the real value matrix are both prior art, and will not be described herein.

[0135] The real key matrix and the real value matrix corresponding to the plurality of original training samples can be stored in a preset position. For each original training sample, the server obtains the real key matrix and the real value matrix corresponding to the original training sample from the preset position, then determines a first key-related loss according to the key matrix and the real key matrix, and determines a first value-related loss according to the value matrix and the real value matrix; and finally, the first key-related loss and the first value-related loss determine the first attention loss corresponding to the original training sample.

[0136] In this way, the first key-related loss and the first value-related loss respectively measure the difference between the key matrix and the value matrix and their real values, and the first attention loss corresponding to the original training sample is determined according to the first key-related loss and the first value-related loss, which can improve the accuracy of the determined first attention loss.

[0137] Optionally, the first key-related loss is determined according to the key matrix and the real key matrix, and the first value-related loss is determined according to the value matrix and the real value matrix, comprising:

[0138] calculating the first mutual information of the key matrix and the real key matrix, and determining the first key-related loss according to the first mutual information;

[0139] calculating the second mutual information of the value matrix and the real value matrix, and determining the first value-related loss according to the second mutual information.

[0140] Mutual information is an important concept in information theory, which is used to measure the dependence between two random variables or the degree of information sharing. Specifically, mutual information quantifies the amount of information contained in one random variable about another random variable.

[0141] For example, the key matrix is represented by K f , the real key matrix is represented by K g , I(K g ; K f ) is the mutual information degree between the key matrix and the real key matrix, which measures the degree of K g containing K f .

[0142] The value matrix is represented by V f , the real value matrix is represented by V g , and I(V g ; V f ) is the mutual information degree between the value matrix and the real value matrix, which measures the degree of V g containing V f .

[0143] Specifically, the first mutual information of the key matrix and the real key matrix is calculated, and a first key-related loss is determined according to the first mutual information; the second mutual information of the value matrix and the real value matrix is calculated, and a second key-related loss is determined according to the second mutual information.

[0144] In this way, the mutual information quantifies the dependency and information sharing degree between two matrices, thereby improving the accuracy of determining the first key-related loss and the first value-related loss.

[0145] Optionally, the first key-related loss is determined according to the first mutual information, including:

[0146] A first Pearson correlation coefficient of the key matrix and the real key matrix is calculated.

[0147] The first mutual information and the first Pearson correlation coefficient are added to obtain a first sum.

[0148] Half of the first sum is determined as the first key-related loss.

[0149] Correspondingly, the first value-related loss is determined according to the second mutual information, including:

[0150] A second Pearson correlation coefficient of the value matrix and the real value matrix is calculated.

[0151] The second mutual information and the second Pearson correlation coefficient are added to obtain a second sum.

[0152] Half of the second sum is determined as the first value-related loss.

[0153] The Pearson correlation coefficient is also called Pearson product-moment correlation coefficient, which is a statistical quantity used to measure the degree of linear correlation between two variables.

[0154] Specifically, is the Pearson correlation coefficient between the key matrix and the real key matrix, used to measure the linear correlation between the key matrix and the real key matrix. is the Pearson correlation coefficient between the value matrix and the real value matrix, used to measure the linear correlation between the value matrix and the real value matrix.

[0155] Specifically, the server calculates the first Pearson correlation coefficient of the key matrix and the real key matrix The first mutual information I(K g ;K f ) and the first Pearson correlation coefficient are added to obtain a first sum: The first key-related loss is

[0156] Similarly, the server calculates a second Pearson correlation coefficient of the value matrix and the real value matrix The second mutual information I(V g ; V f ) and the second Pearson correlation coefficient are added to obtain a second sum: The first value correlation loss is

[0157] In this way, the mutual information and the Pearson correlation coefficient measure the dependency between the matrices from different angles, respectively. By combining the two measurement methods, the accuracy of the determined first key correlation loss and first value correlation loss can be further improved.

[0158] Optionally, the attention loss includes a second attention loss; after the original training sample is input into the attention model, a key matrix and a value matrix are obtained, and the attention loss corresponding to the original training sample is determined according to the key matrix and the value matrix, including:

[0159] After the original training sample is input into the attention model, a key matrix, a value matrix and a query matrix are obtained;

[0160] The second attention loss corresponding to the original training sample is determined according to the key matrix, the value matrix and the query matrix.

[0161] The query matrix is obtained by linearly transforming the input sequence corresponding to the original training sample, and can represent the input feature of the current step of the decoder, and is used to calculate the attention weight with the key matrix.

[0162] Specifically, for each original training sample, the server inputs the original training sample into the encoder of the attention model, and the encoder outputs a key matrix, a value matrix and a query matrix. The attention loss corresponding to the original training sample is determined according to the key matrix, the value matrix and the query matrix output by the encoder.

[0163] In this way, the attention loss is determined based on the key matrix, the value matrix and the query matrix, which can further improve the accuracy of the determined attention loss.

[0164] Optionally, the second attention loss corresponding to the original training sample is determined according to the key matrix, the value matrix and the query matrix, including:

[0165] The real key matrix, the real value matrix and the real query matrix corresponding to the original training sample are obtained;

[0166] The second key correlation loss is determined according to the key matrix and the real key matrix;

[0167] determine a second value-related loss according to the value matrix and the real value matrix;

[0168] determine a query-related loss according to the query matrix and the real query matrix;

[0169] determine the second attention loss corresponding to the original training sample according to the second key-related loss, the second value-related loss and the query-related loss.

[0170] The method for determining the real key matrix, the real value matrix and the real query matrix is prior art, which will not be described herein.

[0171] Specifically, the server determines the second key-related loss according to the key matrix and the real key matrix, determines the second value-related loss according to the value matrix and the real value matrix, determines the query-related loss according to the query matrix and the real query matrix, and determines the second attention loss corresponding to the original training sample according to the second key-related loss, the second value-related loss and the query-related loss.

[0172] In the present application, the methods for calculating the second key-related loss, the second value-related loss and the query-related loss are the same.

[0173] In this way, the key-related loss, the value-related loss and the query-related loss are calculated respectively, which can more comprehensively measure the difference between the matrix generated by the model and the real matrix, and further improve the accuracy of the determined second attention loss.

[0174] Optionally, the real key matrix, the real value matrix and the real query matrix each include a plurality of elements, for each element, if an element value corresponding to the element is a first preset value, it indicates that a position where the element is located should be paid attention to, and if the element value corresponding to the element is a second preset value, it indicates that the position where the element is located should not be paid attention to.

[0175] For example, in the image field, a region of a certain image that needs to be paid attention to is a square with a size of 4*4 (pixels) in the center of the image, the first preset value is 1, and the second preset value is 0. FIG. 3 is a schematic diagram of a real key matrix provided by an embodiment of the present application, as shown in FIG. 3, the element values of the elements in the central 4*4 region of the matrix are all 1, and the element values of the remaining elements are all 0. For any element, if the element value corresponding to the element is 1, it indicates that the position where the element is located should be paid attention to, and if the element value corresponding to the element is 0, it indicates that the position where the element is located should not be paid attention to.

[0176] The real value matrix and the real query matrix can be the same as the real key matrix.

[0177] In this way, by using the preset value to mark the positions that should be paid attention to and the positions that should not be paid attention to, the real distribution of attention can be more accurately reflected. And it is beneficial to subsequent numerical calculation, and improves the operation efficiency.

[0178] Optionally, the second key-related loss is determined according to the key matrix and the real key matrix, including:

[0179] The element values corresponding to the elements in the key matrix are scaled to between the first preset value and the second preset value to obtain a scaled key matrix.

[0180] The second key-related loss is determined according to the scaled key matrix and the real key matrix.

[0181] Specifically, the server scales the element values corresponding to the elements in the key matrix to between the first preset value and the second preset value, such as between 0 and 1, to obtain a scaled key matrix. In the scaled key matrix, the element values corresponding to each element are between the first preset value and the second preset value. The second key-related loss is determined according to the scaled key matrix and the real key matrix.

[0182] In this way, scaling the element values of the key matrix to between the first preset value and the second preset value helps to avoid numerical overflow or underflow, and enhances the numerical stability in the calculation process. Determining the second key-related loss according to the scaled key matrix and the real key matrix can improve the accuracy of the determined second key-related loss.

[0183] Optionally, the second key-related loss is determined according to the scaled key matrix and the real key matrix, including:

[0184] For each element in the scaled key matrix, the difference between the element value corresponding to the element and the element value corresponding to a target element is calculated. The relative position of the target element in the real key matrix is the same as the relative position of the element in the key matrix.

[0185] The sum of the difference values corresponding to the elements in the scaled key matrix is calculated to obtain a third sum.

[0186] The second key-related loss is determined according to the third sum.

[0187] Exemplarily, the scaled key matrix and the real key matrix are both 3*3 matrices, for an element in the first row and the first column of the scaled key matrix, a difference d1 between the element and a target element in the first row and the first column of the real key matrix is calculated, for an element in the first row and the second column of the scaled key matrix, a difference d1 between the element and a target element in the first row and the second column of the real key matrix is calculated, and so on, for an element in the third row and the third column of the scaled key matrix, a difference d9 between the element and a target element in the third row and the third column of the real key matrix is calculated, and a third sum is equal to d1+d2+…+d9, and the second key-related loss is determined according to the third sum.

[0188] In this way, by calculating the difference between the scaled key matrix and the real key matrix element by element, the error of each element can be accurately captured, thereby improving the accuracy of loss calculation, and the difference is accumulated into the third sum element by element, which simplifies the loss calculation process and makes the calculation of the loss function more intuitive and easy to implement.

[0189] Optionally, the second key-related loss is determined according to the third sum, including:

[0190] The intersection over union of the key matrix and the real key matrix is calculated,

[0191] The product of the intersection over union and the third sum is determined as the second key-related loss.

[0192] Wherein, the intersection over union (IOU) of two matrices is usually used to evaluate the degree of overlap between two matrices.

[0193] Specifically, the server calculates the intersection over union of the key matrix and the real key matrix, and determines the product of the intersection over union and the third sum as the second key-related loss.

[0194] In this way, by calculating the intersection over union of the key matrix and the real key matrix, the degree of overlap between the two matrices can be evaluated. By combining the intersection over union with the third sum, the difference between the key matrix and the real key matrix can be more comprehensively evaluated, thereby obtaining a more accurate loss value.

[0195] Optionally, after the original training sample is input into the attention model, the key matrix and the value matrix are obtained, and the attention loss corresponding to the original training sample is determined according to the key matrix and the value matrix, including:

[0196] After the original training sample is input into the attention model, the key matrix, the value matrix and the query matrix are obtained;

[0197] The first attention loss is determined based on the key matrix and the value matrix;

[0198] determine a second attention loss based on the key matrix, the value matrix and the query matrix;

[0199] determine the attention loss corresponding to the original training sample based on the first attention loss and the second attention loss.

[0200] Specifically, the server inputs the original training sample into the encoder of the attention model to obtain the key matrix, the value matrix and the query matrix; based on the key matrix and the value matrix, the first attention loss can be determined, and the specific determination method can refer to any of the above embodiments, which will not be described here; based on the key matrix, the value matrix and the query matrix, the second attention loss can be determined, and the specific determination method can also refer to any of the above embodiments, which will not be described here.

[0201] The determination of the attention loss corresponding to the original training sample based on the first attention loss and the second attention loss can be in various ways, which are not limited by the present application.

[0202] In an optional implementation, the first attention loss and the second attention loss are added to obtain the attention loss corresponding to the original training sample.

[0203] In another optional implementation, the server first obtains the weight α corresponding to the first attention loss and the weight β corresponding to the second attention loss, and the attention loss corresponding to the original training sample = the first attention loss x α + the second attention loss x β.

[0204] In this way, by calculating the first attention loss based on the key matrix and the value matrix and the second attention loss based on the key matrix, the value matrix and the query matrix respectively, the attention loss can be evaluated from multiple dimensions, so as to improve the accuracy of the determined attention loss corresponding to the original training sample.

[0205] Step 203, determine a first total attention loss according to the attention loss corresponding to each original training sample respectively.

[0206] The sum of the attention loss corresponding to each original training sample respectively is the first total attention loss.

[0207] Step 204, determine a second total attention loss based on a plurality of target training samples and an attention model; the target training sample includes at least one first feature label and a feature value corresponding to each first feature label; the first feature label is selected from a plurality of original feature labels included in the original training sample.

[0208] Specifically, for each original training sample, the server screens at least one first feature label from the plurality of original feature labels corresponding to the original training sample, and the at least one first feature label and the feature values corresponding to each first feature label constitute a target training sample. The screening process can be random screening or screening according to a preset rule.

[0209] The server inputs the plurality of target training samples into the attention model respectively to determine a second total attention loss. The method of determining the second total attention loss is the same as the method of determining the first total attention loss, and is specifically as follows:

[0210] For each target training sample, the target training sample is input into the encoder in the attention model to obtain a corresponding key matrix and value matrix, and the attention loss corresponding to the target training sample is determined according to the obtained key matrix and value matrix. The method of determining the attention loss corresponding to the target training sample according to the obtained key matrix and value matrix is the same as the method of determining the attention loss corresponding to the original sample according to the key matrix and value matrix, and can refer to the method in any of the above embodiments.

[0211] The server adds the attention losses corresponding to the target training samples respectively to obtain a total sum as the second total attention loss.

[0212] Step 205, determining whether to delete a second feature label according to the first total attention loss and the second total attention loss, wherein the second feature label is a feature label other than the at least one first feature label in the plurality of original feature labels.

[0213] Specifically, when the difference between the first total attention loss and the second total attention loss is less than a preset threshold, it is determined to delete the second feature label.

[0214] When the difference between the first total attention loss and the second total attention loss is greater than the preset threshold, it is determined not to delete the second feature label. Further, at least one third feature label is selected from the second feature label, and for each original training sample, the corresponding third feature label and the first feature label of the original sample are combined to obtain a first target training sample. The first target training sample is input into the attention model to determine a corresponding third attention loss. When the difference between the third attention loss and the first total attention loss is less than the preset threshold, the fourth feature label other than the third feature label in the second feature label is deleted. When the difference between the third attention loss and the first total attention loss is greater than the preset threshold, the fourth feature label is not deleted.

[0215] Through the above method, it can be determined whether each original feature label is deleted one by one.

[0216] For example, the original feature labels corresponding to the original training samples include: age, gender, marital status, total assets, total liabilities, expected record, default record, transaction amount, and transaction frequency of the user.

[0217] The first feature labels screened out include: age, gender, marital status, total assets, and total liabilities of the user. The first feature labels and the feature values corresponding to each first feature label constitute the target training samples.

[0218] The first total attention loss is calculated according to each original training sample, the second total attention loss is calculated according to each target training sample, when the difference between the first total attention loss and the second total attention loss is less than a preset difference value, it is determined that the second feature label is deleted, that is, the labels: expected record, default record, transaction amount, and transaction frequency are deleted.

[0219] When the difference between the first total attention loss and the second total attention loss is greater than the preset difference value, it is determined that the second feature label is not deleted, and the third feature label is screened out from the second feature label, that is, the expected record and the default record.

[0220] The first target training sample is obtained by combining the third feature label with the first feature label, and the labels corresponding to the first target training sample are: age, gender, marital status, total assets, total liabilities, expected record, and default record of the user.

[0221] The third total attention loss is calculated according to the first target training sample, when the difference between the first total attention loss and the third total attention loss is less than a preset difference value, it is determined that the labels transaction amount and transaction frequency are deleted, and when the difference between the first total attention loss and the third total attention loss is greater than the preset difference value, it is determined that the labels transaction amount and transaction frequency are not deleted.

[0222] The feature label screening method provided in the embodiments of the present application comprises the following steps: obtaining a plurality of original training samples, each original training sample comprising a plurality of original feature labels and feature values corresponding to each original feature label; for each original training sample, obtaining a key matrix and a value matrix after the original training sample is input into an attention model, and determining an attention loss corresponding to the original training sample according to the key matrix and the value matrix, wherein the attention model comprises an encoder and a decoder, and the key matrix and the value matrix are transmitted from the encoder to the decoder; determining a first total attention loss according to the attention loss corresponding to each original training sample; determining a second total attention loss based on a plurality of target training samples and the attention model; the target training sample comprises at least one first feature label and feature values corresponding to each first feature label; the first feature label is screened from the plurality of original feature labels comprised in the original training sample; and determining whether to delete a second feature label according to the first total attention loss and the second total attention loss, wherein the second feature label is a feature label other than the at least one first feature label in the plurality of original feature labels. The present application creatively takes the attention loss as an index for measuring the importance of a feature label, which can effectively improve the accuracy of feature label screening, and by screening the most important feature label, the interference of noise and redundant information can be reduced, the model can be more focused on key features, and the generalization ability and prediction accuracy of the model can be improved.

[0223] Optionally, the feature label screening method provided in the embodiments of the present application further comprises:

[0224] obtaining a sample label corresponding to each original training sample;

[0225] for each original training sample, determining an output corresponding to the original training sample, and determining an output loss corresponding to the original training sample according to the sample label and the output corresponding to the original training sample;

[0226] determining a first total output loss according to the output loss corresponding to each original training sample;

[0227] for each target training sample, determining an output corresponding to the target training sample, and determining an output loss corresponding to the target training sample according to the sample label and the output corresponding to the target training sample;

[0228] determining a second total output loss according to the output loss corresponding to each target training sample;

[0229] Correspondingly, determining whether to delete the second feature label according to the first total attention loss and the second total attention loss comprises:

[0230] According to the first total attention loss, the second total attention loss, the first total output loss, and the second total output loss, it is determined whether to delete the second feature label.

[0231] The target training sample has the same sample label as the corresponding original training sample.

[0232] Specifically, the server obtains a sample label corresponding to each original training sample; for each original training sample, the server inputs the original training sample into the attention model, and the attention model outputs an output corresponding to the original training sample; and the server determines an output loss corresponding to the original training sample according to the sample label and the output corresponding to the original training sample.

[0233] The output losses corresponding to the original training samples are summed up, and a sum obtained is determined as a first total output loss.

[0234] For each target training sample, the server inputs the target training sample into the attention model, and the attention model outputs an output corresponding to the target training sample; and the server determines an output loss corresponding to the target training sample according to the sample label and the output corresponding to the target training sample.

[0235] The output losses corresponding to the target training samples are summed up, and a sum obtained is determined as a second total output loss.

[0236] Finally, the server determines whether to delete the second feature label according to the first total attention loss, the second total attention loss, the first total output loss, and the second total output loss.

[0237] In this way, by calculating the output losses and the attention losses of the original training samples and the target training samples, the importance of the feature label can be comprehensively evaluated from more angles, thereby improving the accuracy of determining whether to delete the second feature label.

[0238] Optionally, according to the first total attention loss, the second total attention loss, the first total output loss, and the second total output loss, it is determined whether to delete the second feature label, including:

[0239] According to the first total attention loss and the first total output loss, a first total loss is determined.

[0240] According to the second total attention loss and the second total output loss, a second total loss is determined.

[0241] When a difference between the second total loss and the first total loss is less than a preset threshold, it is determined that the second feature label is deleted.

[0242] Specifically, the server calculates a sum of the first total attention loss and the first total output loss, determines a result of the sum as the first total loss; calculates a sum of the second total attention loss and the second total output loss, determines a result of the sum as the second total loss; determines to delete the second feature label when a difference between the second total loss and the first total loss is less than a preset threshold, and determines not to delete the second feature label when the difference between the second total loss and the first total loss is greater than the preset threshold.

[0243] In this way, by combining the attention loss and the output loss, the importance of the feature label can be more accurately evaluated, and by the preset threshold, it is automatically determined whether to delete the feature label, reducing the necessity of human intervention, making the feature selection process more automated and efficient.

[0244] Optionally, the first total loss is determined according to the first total attention loss and the first total output loss, including:

[0245] The first weight corresponding to the first total attention loss and the second weight corresponding to the first total output loss are obtained.

[0246] The first total loss is determined according to the first total attention loss, the first total output loss, the first weight and the second weight.

[0247] Specifically, the server obtains the first weight a corresponding to the first total attention loss and the second weight b corresponding to the first total output loss, and the first total loss = the first total attention loss x a + the first total output loss x b.

[0248] In this way, the introduction of the weight parameters (the first weight and the second weight) allows the different types of losses to be weighted and adjusted. In this way, according to the requirements of specific tasks, the contribution proportion of the attention loss and the output loss in the total loss can be flexibly adjusted, thereby improving the accuracy of determining the first total loss.

[0249] Optionally, the second total loss is determined according to the second total attention loss and the second total output loss, including:

[0250] The third weight corresponding to the second total attention loss and the fourth weight corresponding to the second total output loss are obtained.

[0251] The second total loss is determined according to the second total attention loss, the second total output loss, the third weight and the fourth weight.

[0252] Optionally, the output corresponding to each original training sample is a classification probability value, the classification probability value is used to indicate a probability of the original training sample being divided into each category of a plurality of preset categories, and the output loss corresponding to the original training sample is determined according to the sample label and the output corresponding to the original training sample, including:

[0253] determining the output loss corresponding to the original training sample according to the following formula:

[0254] wherein, l represents the output loss, M represents the total number of preset multiple categories, C represents a category, P c represents the probability of being divided into the Cth category, y c represents the sample label, and γ is a constant greater than 1.

[0255] Specifically, the index γ is a loss shrinkage factor, which is used to obtain a more accurate output loss under the same conditions. FIG. 4 is a schematic diagram of the relationship between the output loss and the classification probability value according to an embodiment of the present application. As shown in FIG. 4, the presence of γ is used to "tighten" the loss function, and with the increase of γ, the loss value is smaller under the same independent variable condition, that is, a small value is more conducive to numerical operation than a large value.

[0256] In this way, by adding the loss shrinkage factor, the accuracy of the determined output loss can be improved.

[0257] Optionally, the output corresponding to each original training sample is a classification probability value, the classification probability value is used to indicate the probability of the original training sample being divided into each category of the preset multiple categories, and the output loss corresponding to the original training sample is determined according to the sample label and the output corresponding to the original training sample, including:

[0258] determining the output loss corresponding to the original training sample according to the following formula:

[0259] wherein, l represents the output loss, M represents the total number of preset multiple categories, C represents a category, P c represents the probability of being divided into the Cth category, y c represents the sample label, and γ is a constant greater than 1.

[0260] In this way, for On the basis of the index γ, the P c term is further added, when the probability corresponding to a certain category is larger, the overall index term is larger, and the loss value is smaller, that is, the loss caused by the greater possibility of being correct is smaller, because the accuracy of the determined output loss can be further improved.

[0261] FIG. 5 is another feature label screening method provided by an embodiment of the present application. The execution subject of the present embodiment can be any device with data processing function, and the present application takes the client as the execution subject for specific description. As shown in FIG. 5, the another feature label screening method provided by an embodiment of the present application can include:

[0262] Step 501, obtaining a plurality of local training samples, each local training sample comprising a plurality of local feature labels and feature values corresponding to each local feature label.

[0263] Specifically, the client obtains a plurality of local training samples, each local training sample comprising a plurality of local feature labels and feature values corresponding to each local feature label.

[0264] Step 502, for each local training sample, encrypting the feature values of each local feature label corresponding to the local training sample to obtain an original training sample, wherein the encryption method is to add Laplace noise.

[0265] Specifically, for each local training sample, the client encrypts the feature values of each local feature label corresponding to the local training sample to obtain an original training sample, wherein the encryption method is to add Laplace noise.

[0266] Step 503, sending a plurality of original training samples to the server, each original training sample comprising a plurality of original feature labels and feature values corresponding to each original feature label; so that the server, for each original training sample, obtains a key matrix and a value matrix after the original training sample is input into an attention model, and determines an attention loss corresponding to the original training sample according to the key matrix and the value matrix, the attention model comprising an encoder and a decoder, the key matrix and the value matrix being passed from the encoder to the decoder; determining a first total attention loss according to the attention loss corresponding to each original training sample respectively; determining a second total attention loss based on a plurality of target training samples and the attention model; the target training sample comprising at least one first feature label and feature values corresponding to each first feature label; the first feature label being selected from the plurality of original feature labels comprised in the original training sample; determining whether to delete a second feature label according to the first total attention loss and the second total attention loss, wherein the second feature label is a feature label other than the at least one first feature label in the plurality of original feature labels.

[0267] Specifically, the client sends a plurality of original training samples to the server, the server receives the plurality of original training samples, for each original training sample, inputs the original training sample into an attention model to obtain a key matrix and a value matrix, and determines an attention loss corresponding to the original training sample according to the key matrix and the value matrix, the attention model includes an encoder and a decoder, and the key matrix and the value matrix are transmitted by the encoder to the decoder; a first total attention loss is determined according to the attention loss corresponding to each original training sample; a second total attention loss is determined based on a plurality of target training samples and the attention model; the target training sample includes at least one first feature label and a feature value corresponding to each first feature label; the first feature label is selected from a plurality of original feature labels included in the original training sample; and whether to delete a second feature label is determined according to the first total attention loss and the second total attention loss, wherein the second feature label is a feature label other than the at least one first feature label in the plurality of original feature labels.

[0268] In this way, the client encrypts the feature values of each local feature label corresponding to the local training sample to obtain the original training sample, and then sends it to the server, which can improve the security of the data and reduce the risk of important information leakage. And the application creatively uses attention loss as an indicator to measure the importance of feature labels, which can effectively improve the accuracy of feature label screening, and by screening the most important feature labels, it can reduce the interference of noise and redundant information, make the model more focused on key features, and improve the generalization ability and prediction accuracy of the model.

[0269] Optionally, the method further comprises:

[0270] normalizing the feature values corresponding to each local feature label to a preset dimension;

[0271] Correspondingly, the feature values of each local feature label corresponding to the local training sample are encrypted to obtain the original training sample, including:

[0272] The normalized feature values corresponding to the local training sample are encrypted to obtain the original training sample.

[0273] Specifically, the client normalizes the feature values corresponding to each local feature label to a preset dimension, and then encrypts each normalized feature value to obtain the original training sample.

[0274] In this way, normalizing the feature values corresponding to each local feature label to a preset dimension helps to eliminate the dimensional difference between different features, so that the feature values are compared and processed on the same scale, and the adverse effects caused by some feature values being too large or too small can be avoided.

[0275] The application embodiment also provides another feature label screening method. The specific steps are as follows:

[0276] Step one: Based on the longitudinal federal learning scene, each institution (i.e. multiple clients) aligns the unified customer group.

[0277] Step two: Based on the unsupervised clustering and PCA dimension reduction of the initial feature label of each institution, redundant feature labels and redundant samples are screened out. FIG. 6 is a customer group feature label clustering and dimension reduction schematic diagram provided by the application embodiment, as shown in FIG. 6, which shows that after the feature clustering and dimension reduction, the feature dimension is 3 (the feature labels include: age, card type and the maximum single consumption amount in the last six months), and the customer group can be generally divided.

[0278] Step three: For the initial feature label screened out in the previous step, the feature value corresponding to the initial feature label is normalized to a specific dimension to avoid large numerical features (such as large transaction flow features) in subsequent calculations.

[0279] Step four: For each institution, the normalized feature value corresponding to the institution is contributed in the form of a numerical vector.

[0280] For example, for the feature values corresponding to the age feature, the maximum single consumption amount in the last six months, and the card type in FIG. 6, they are uniformly normalized to the interval of 0-5, let d=10, then: age / d, maximum single consumption amount in the last six months / d, and card type is marked as: 0-no credit card under the name, 1-card type 1, 2-card type 2, 3-card type 3. For customer A who is 25 years old, has a maximum single consumption amount of 50,000 yuan in the last six months, and has a card type 1, the normalized plaintext feature vector of the three feature vectors of A bank is F A o=[2.5,5,1], and the encrypted ciphertext feature vector is F A e=[lap(2.5),lap(5),lap(1)]. Similarly, assuming that the maximum single consumption amount of customer A in B bank in the last six months is 70,000 yuan, and there is no credit card, then the plaintext vector is F B o=[2.5,7,0], and the ciphertext vector is F B e=[lap(2.5),lap(7),lap(0)]. When A bank and B bank jointly train the financial big model in the longitudinal federal scene, the de-sensitized features F A e and F B e are the out-of-domain features contributed by the two banks.

[0281] Step five: After the server receives the numerical vectors sent by each agency, it performs federated feature fusion and needs to repeat the unsupervised clustering -> dimensionality reduction in step two before inputting into the large model.

[0282] Step six: For the K, V attention matrix passed from the encoder of the large model to the decoder, calculate the "attention drift loss";

[0283] The specific "attention drift loss" function is designed as follows:

[0284] (1) Exponential cross-entropy loss: the output end cross-entropy loss of the large model decoder is used as the overall detection index of the model, which intuitively reflects the influence of input features on output from the perspective of output accuracy, denoted as:

[0285] Where γ is the loss shrinkage factor, used to obtain more accurate loss under the same conditions, and (1-P ic ) x The existence of the exponential x of the term is used to "tighten" the loss function, and the larger x is, the smaller the loss value is under the same independent variable condition, i.e. small values are more conducive to numerical operations than large values. Further, the P ic term is added to the exponential x, and P is the classification probability value when the sample i is of class C. When the probability of a certain class is large, the overall exponential term is larger, and the loss value is smaller, i.e. "the greater the correct possibility, the smaller the loss".

[0286] (2) Attention matrix loss: in the attention model (which can be a transformer model), the output of the encoder stack is the K (key matrix) and V (value matrix) in the attention matrix. K, V are the input of each decoder in the decoder stack, so the loss function here measures the capture ability of all encoders to the original input. Denote the real attention matrix as A g , the drifted attention matrix as A f (the key matrix or value matrix output by the encoder), and the attention matrix loss as:

[0287] Where I(A g ; A f ) is the mutual information measure between the drifted attention matrix and the real attention matrix, which measures the degree to which A g contains A f information. is the Pearson correlation coefficient between the drifted attention matrix and the ground truth (real) attention matrix, used to measure the linear correlation between A g and A f .

[0288] Calculate the key-related attention loss between the key matrix and the true key matrix, and the value-related attention loss between the value matrix and the true value matrix, using the formulas described above. Adding the calculated key-related attention loss to the value-related attention loss gives the overall attention loss for both the key matrix and the value matrix.

[0289] (3) Ground Truth Masked IOU Loss: Borrowing the traditional "mask" concept, this proposal first suggests a ground truth masked attention matrix, which represents the position where the true attention should exist in the matrix. Figure 7 shows a ground truth attention matrix provided by an embodiment of this application. As shown in Figure 7, the area where the elements that should be paid attention are located constitutes the cat's head.

[0290] Secondly, during model training, we perform training on each position Q of the Q, K, and V matrices. i,j K i,j V i,j First, scale the matrix (to a value between 0 and 1), then apply the Q-value to each position of the ground truth mask attention matrix. gi,j K gi,j V gi,j The difference between the Q, K, and V attention matrices is denoted uniformly as A. i,j A gi,j Figure 8 is a schematic diagram of a difference calculation provided in an embodiment of this application. As shown in Figure 8, the element Q in the fourth row and fifth column of the query matrix is... 4,5 =0.7 and the actual query matrix Q g4,5 The difference between 1 and 0.3 is 0.

[0291] Next, we calculate the IOU value between the ground truth mask attention matrix and the Q, K, and V matrices generated during model training, denoted as I. q I k I v (The attention matrices Q, K, and V are uniformly denoted as I) A IOU≈0.5.

[0292] Figure 9 is a schematic diagram of image attention drift and IOU provided in an embodiment of this application. As shown in Figure 9, the area within the blue border is the attention area determined by the model, the area within the green border is the actual attention area, the area within the red border is the intersection area of ​​the two attention areas, and the area within the black border is the union area of ​​the two attention areas.

[0293] Based on the above description of the true value masking loss, the expression for this loss term is designed as follows:

[0294] l1=I A ·∑|A i,j -A gi,j |

[0295] The overall model training loss function:

[0296] Wherein, a, b, g are balance weight coefficients, s is a scaling factor (because the l3 term loss is scaled to 0-1 to calculate the difference with the true attention matrix, and now the overall loss calculation needs to be consistent in numerical value with the other two terms).

[0297] Step seven: feedback the "attention drift loss" to the input feature engineering to determine whether to delete the feature label of the large model input.

[0298] The application also provides a feature label screening system, the system comprises:

[0299] The server is used for executing the method described in any of the above embodiments;

[0300] The client is used for executing the method described in any of the above embodiments.

[0301] The specific implementation principle and effect of the feature label screening system provided by the embodiments of the application can be referred to the foregoing embodiments, which will not be described here.

[0302] Corresponding to the above feature label screening method, the embodiments of the application also provide a feature label screening device, and FIG. 10 is a structural schematic diagram of a feature label screening device provided by an embodiment of the application. As shown in FIG. 10, the device comprises:

[0303] The acquisition module 1001 is used for acquiring a plurality of original training samples, each original training sample comprising a plurality of original feature labels and feature values corresponding to each original feature label;

[0304] The first determination module 1002 is used for, for each original training sample, obtaining a key matrix and a value matrix after the original training sample is input into an attention model, and determining an attention loss corresponding to the original training sample according to the key matrix and the value matrix, wherein the attention model comprises an encoder and a decoder, and the key matrix and the value matrix are transmitted by the encoder to the decoder;

[0305] The second determination module 1003 is used for determining a first total attention loss according to the attention loss corresponding to each original training sample respectively;

[0306] The third determination module 1004 is used for determining a second total attention loss based on a plurality of target training samples and an attention model, wherein the target training samples comprise at least one first feature label and feature values corresponding to each first feature label; the first feature label is screened from the plurality of original feature labels comprised in the original training samples.

[0307] The fourth determining module 1005 is configured to determine whether to delete a second feature label according to the first total attention loss and the second total attention loss, wherein the second feature label is a feature label in the plurality of original feature labels except for the at least one first feature label.

[0308] Optionally, the attention loss includes a first attention loss, and the first determining module 1002, when determining the attention loss corresponding to the original training sample according to the key matrix and the value matrix, is specifically configured to:

[0309] obtain a real key matrix and a real value matrix corresponding to the original training sample;

[0310] determine a first key-related loss according to the key matrix and the real key matrix, and determine a first value-related loss according to the value matrix and the real value matrix;

[0311] determine the first attention loss corresponding to the original training sample according to the first key-related loss and the first value-related loss.

[0312] Optionally, the first determining module 1002, when determining the first key-related loss according to the key matrix and the real key matrix, and determining the first value-related loss according to the value matrix and the real value matrix, is specifically configured to:

[0313] calculate a first mutual information of the key matrix and the real key matrix, and determine the first key-related loss according to the first mutual information;

[0314] calculate a second mutual information of the value matrix and the real value matrix, and determine the first value-related loss according to the second mutual information.

[0315] Optionally, the first determining module 1002, when determining the first key-related loss according to the first mutual information, is specifically configured to:

[0316] calculate a first Pearson correlation coefficient of the key matrix and the real key matrix;

[0317] add the first mutual information and the first Pearson correlation coefficient to obtain a first sum;

[0318] determine half of the first sum as the first key-related loss;

[0319] Correspondingly, determining the first value-related loss according to the second mutual information includes:

[0320] calculating a second Pearson correlation coefficient of the value matrix and the real value matrix;

[0321] adding the second mutual information and the second Pearson correlation coefficient to obtain a second sum;

[0322] determining a half of the second sum as the first value-related loss.

[0323] Optionally, the attention loss includes a second attention loss; the first determining module 1002 is specifically used for:

[0324] after the original training sample is input into the attention model, a key matrix, a value matrix and a query matrix are obtained;

[0325] determining the second attention loss corresponding to the original training sample according to the key matrix, the value matrix and the query matrix.

[0326] Optionally, when determining the second attention loss corresponding to the original training sample according to the key matrix, the value matrix and the query matrix, the first determining module 1002 is specifically used for:

[0327] obtaining a real key matrix, a real value matrix and a real query matrix corresponding to the original training sample;

[0328] determining a second key-related loss according to the key matrix and the real key matrix;

[0329] determining a second value-related loss according to the value matrix and the real value matrix;

[0330] determining a query-related loss according to the query matrix and the real query matrix;

[0331] determining the second attention loss corresponding to the original training sample according to the second key-related loss, the second value-related loss and the query-related loss.

[0332] Optionally, the real key matrix, the real value matrix and the real query matrix each include a plurality of elements, for each element, if an element value corresponding to the element is a first preset value, it indicates that a position where the element is located should be paid attention to, and if the element value corresponding to the element is a second preset value, it indicates that the position where the element is located should not be paid attention to.

[0333] Optionally, when determining the second key-related loss according to the key matrix and the real key matrix, the first determining module 1002 is specifically used for:

[0334] scaling element values corresponding to elements in the key matrix to between the first preset value and the second preset value to obtain a scaled key matrix;

[0335] determining the second key-related loss according to the scaled key matrix and the real key matrix.

[0336] Optionally, the first determining module 1002, in determining the second key-related loss according to the scaled key matrix and the real key matrix, is specifically configured to:

[0337] for each element in the scaled key matrix, calculate a difference value between an element value corresponding to the element and an element value corresponding to a target element, wherein a relative position of the target element in the real key matrix is same as a relative position of the element in the key matrix;

[0338] calculate a sum of the difference values corresponding to the elements in the scaled key matrix respectively to obtain a third sum;

[0339] determine the second key-related loss according to the third sum.

[0340] Optionally, the first determining module 1002, in determining the second key-related loss according to the third sum, is specifically configured to:

[0341] calculate an intersection over union of the key matrix and the real key matrix,

[0342] determine a product of the intersection over union and the third sum as the second key-related loss.

[0343] Optionally, the first determining module 1002 is specifically configured to:

[0344] obtain a key matrix, a value matrix and a query matrix after the original training sample is input into the attention model;

[0345] determine a first attention loss based on the key matrix and the value matrix;

[0346] determine a second attention loss based on the key matrix, the value matrix and the query matrix;

[0347] determine an attention loss corresponding to the original training sample based on the first attention loss and the second attention loss.

[0348] Optionally, the obtaining module 1001 is further configured to:

[0349] obtain a sample label corresponding to each original training sample;

[0350] for each original training sample, determine an output corresponding to the original training sample, and determine an output loss corresponding to the original training sample according to the sample label and the output corresponding to the original training sample;

[0351] determine a first total output loss according to the output losses respectively corresponding to the original training samples;

[0352] For each target training sample, determine the output corresponding to the target training sample, and determine the output loss corresponding to the target training sample according to the sample label and the output corresponding to the target training sample;

[0353] According to the output loss corresponding to each target training sample, determine a second total output loss;

[0354] Correspondingly, the fourth determination module 1005 is specifically configured to:

[0355] According to the first total attention loss, the second total attention loss, the first total output loss, and the second total output loss, determine whether to delete the second feature label.

[0356] Optionally, the fourth determination module 1005 is specifically configured to:

[0357] According to the first total attention loss and the first total output loss, determine a first total loss;

[0358] According to the second total attention loss and the second total output loss, determine a second total loss;

[0359] When the difference between the second total loss and the first total loss is less than a preset threshold, determine to delete the second feature label.

[0360] Optionally, when the fourth determination module 1005 determines the first total loss according to the first total attention loss and the first total output loss, the fourth determination module 1005 is specifically configured to:

[0361] Obtain a first weight corresponding to the first total attention loss and a second weight corresponding to the first total output loss;

[0362] According to the first total attention loss, the first total output loss, the first weight, and the second weight, determine the first total loss.

[0363] Optionally, the output corresponding to each original training sample is a classification probability value, the classification probability value is used to indicate the probability of the original training sample being divided into each category of a plurality of preset categories, and when the obtaining module 1001 determines the output loss corresponding to the original training sample according to the sample label and the output corresponding to the original training sample, the obtaining module 1001 is specifically configured to:

[0364] According to the following formula, determine the output loss corresponding to the original training sample;

[0365] Wherein, l represents the output loss, M represents the total number of the plurality of preset categories, C represents the category, P c represents the probability of being divided into the Cth category, y c represents the sample label, and γ is a constant greater than 1.

[0366] Optionally, the output corresponding to each original training sample is a classification probability value, the classification probability value is used to indicate a probability that the original training sample is classified into each category in a plurality of preset categories, and the obtaining module 1001 is specifically configured to: determine the output loss corresponding to the original training sample according to the sample label corresponding to the original training sample and the output.

[0367] The output loss corresponding to the original training sample is determined according to the following formula:

[0368] Wherein, l represents the output loss, M represents the total number of the plurality of preset categories, C represents a category, P c represents a probability of being classified into the Cth category, y c represents a sample label, and γ is a constant greater than 1.

[0369] The specific implementation principle and effects of the feature label screening device provided in the embodiments of the present application can be referred to the foregoing embodiments, which will not be described here.

[0370] The embodiments of the present application also provide another feature label screening device, and FIG. 11 is a structural schematic diagram of another feature label screening device provided in the embodiments of the present application. As shown in FIG. 11, the device comprises:

[0371] The obtaining module 1101 is configured to obtain a plurality of local training samples, each local training sample comprising a plurality of local feature labels and feature values corresponding to each local feature label;

[0372] The encryption module 1102 is configured to, for each local training sample, encrypt the feature values of each local feature label corresponding to the local training sample to obtain an original training sample, wherein the encryption mode is to add Laplace noise;

[0373] The sending module 1103 is configured to send a plurality of original training samples to a server, each original training sample comprising a plurality of original feature labels and feature values corresponding to the original feature labels; so that the server, for each original training sample, obtains a key matrix and a value matrix after the original training sample is input into an attention model, and determines an attention loss corresponding to the original training sample according to the key matrix and the value matrix, the attention model comprising an encoder and a decoder, and the key matrix and the value matrix being transmitted by the encoder to the decoder; determines a first total attention loss according to the attention loss corresponding to each original training sample respectively; determines a second total attention loss based on a plurality of target training samples and the attention model; the target training samples comprising at least one first feature label and feature values corresponding to each first feature label; the first feature label being selected from the plurality of original feature labels comprised in the original training samples; and determines whether to delete a second feature label according to the first total attention loss and the second total attention loss, wherein the second feature label is a feature label other than the at least one first feature label in the plurality of original feature labels.

[0374] Optionally, the encryption module 1102 is further configured to:

[0375] normalize the feature values corresponding to each local feature label to a preset dimension;

[0376] Correspondingly, the encryption module 1102, when encrypting the feature values of each local feature label corresponding to the local training sample to obtain the original training sample, is specifically configured to:

[0377] encrypt each normalized feature value corresponding to the local training sample respectively to obtain the original training sample.

[0378] The specific implementation principle and effects of another feature label screening device provided by the embodiment of the application can be referred to the foregoing embodiments, which will not be described here.

[0379] FIG. 12 is a structural schematic diagram of an electronic device provided by the application. As shown in FIG. 12, the electronic device 120 provided by the embodiment of the application comprises at least one processor 1201 and a memory 1202. Optionally, the device 120 further comprises a communication component 1203. Wherein, the processor 1201, the memory 1202 and the communication component 1203 are connected through a bus 1204.

[0380] In the specific implementation process, the at least one processor 1201 executes the computer execution instructions stored in the memory 1202, so that the at least one processor 1201 executes the method described above.

[0381] The specific implementation process of the processor 1201 can refer to the method embodiments described above, which have similar implementation principles and technical effects, and details are not described here again.

[0382] In the above embodiments, it should be understood that the processor can be a central processing unit (English: Central Processing Unit, CPU for short), and can also be other general-purpose processors, digital signal processors (English: Digital Signal Processor, DSP for short), application specific integrated circuits (English: Application Specific Integrated Circuit, ASIC for short), etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in combination with the application can be directly embodied as hardware processor execution, or executed by hardware and software modules in the processor.

[0383] The memory can include a random access memory (RAM), and can also include a non-volatile memory (NVM), such as at least one disk memory.

[0384] The bus can be an industry standard architecture (ISA) bus, a peripheral component (PCI) bus, or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, the bus in the drawings of the present application does not limit only one bus or one type of bus.

[0385] The present application also provides a computer program product, comprising a computer program, which is executed by the processor to realize the above method.

[0386] The present application also provides a computer readable storage medium, which stores computer execution instructions, and when the processor executes the computer execution instructions, the above method is realized.

[0387] The above-mentioned readable storage medium can be realized by any type of volatile or nonvolatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0388] An exemplary readable storage medium is coupled to the processor, so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in the device.

[0389] The division of units is only a logical functional division, and in actual implementation, there can be another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0390] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment.

[0391] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit.

[0392] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the technical solutions that essentially contribute to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the embodiments of the method of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0393] It can be understood by those skilled in the art that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware. The aforementioned program can be stored in a computer readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, and various media that can store program codes.

[0394] Finally, it should be noted that: those skilled in the art will easily think of other embodiments of the present application after considering the specification and practicing the application disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application that follow the general principles of the present application and include common knowledge or conventional technical means in the art that are not disclosed in the present application, and is not limited to the precise structure described above and shown in the drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the present application is only limited by the appended claims.

Claims

1. A feature label screening method, comprising: obtaining a plurality of original training samples, each original training sample comprising a plurality of original feature labels and feature values corresponding to each original feature label; for each original training sample, obtaining a key matrix and a value matrix after the original training sample is input into an attention model, and determining an attention loss corresponding to the original training sample according to the key matrix and the value matrix, wherein the attention model comprises an encoder and a decoder, and the key matrix and the value matrix are transmitted from the encoder to the decoder; determining a first total attention loss according to the attention loss corresponding to each original training sample; determining a second total attention loss based on a plurality of target training samples and the attention model, wherein the target training samples comprise at least one first feature label and feature values corresponding to each first feature label; the first feature label is screened from the plurality of original feature labels comprised in the original training samples; determining whether to delete a second feature label according to the first total attention loss and the second total attention loss, wherein the second feature label is a feature label other than the at least one first feature label in the plurality of original feature labels.

2. The method of claim 1, wherein, the attention loss comprises a first attention loss; determining the attention loss corresponding to the original training sample according to the key matrix and the value matrix comprises: obtaining a real key matrix and a real value matrix corresponding to the original training sample; determining a first key-related loss according to the key matrix and the real key matrix, and determining a first value-related loss according to the value matrix and the real value matrix; determining the first attention loss corresponding to the original training sample according to the first key-related loss and the first value-related loss.

3. The method of claim 2, wherein, determining the first key-related loss according to the key matrix and the real key matrix, and determining the first value-related loss according to the value matrix and the real value matrix comprises: calculating a first mutual information of the key matrix and the real key matrix, and determining the first key-related loss according to the first mutual information; calculating a second mutual information of the value matrix and the real value matrix, and determining the first value-related loss according to the second mutual information.

4. The method of claim 3, wherein, determining the first key-related loss according to the first mutual information comprises: calculating a first Pearson correlation coefficient of the key matrix and the real key matrix; adding the first mutual information and the first Pearson correlation coefficient to obtain a first sum; determining half of the first sum as the first key-related loss; correspondingly, determining the first value-related loss according to the second mutual information comprises: calculating a second Pearson correlation coefficient of the value matrix and the real value matrix; adding the second mutual information and the second Pearson correlation coefficient to obtain a second sum; determining half of the second sum as the first value-related loss.

5. The method of any one of claims 1-4, wherein, the attention loss comprises a second attention loss; and determining the attention loss corresponding to the original training sample according to the key matrix and the value matrix after the original training sample is input into the attention model comprises: After the original training sample is input into the attention model, a key matrix, a value matrix and a query matrix are obtained; According to the key matrix, the value matrix and the query matrix, a second attention loss corresponding to the original training sample is determined.

6. The method of claim 5, wherein, According to the key matrix, the value matrix and the query matrix, the second attention loss corresponding to the original training sample is determined, comprising: Obtaining a real key matrix, a real value matrix and a real query matrix corresponding to the original training sample; According to the key matrix and the real key matrix, a second key-related loss is determined; According to the value matrix and the real value matrix, a second value-related loss is determined; According to the query matrix and the real query matrix, a query-related loss is determined; According to the second key-related loss, the second value-related loss and the query-related loss, the second attention loss corresponding to the original training sample is determined.

7. The method of claim 6, wherein, The real key matrix, the real value matrix and the real query matrix each include a plurality of elements, and for each element, if an element value corresponding to the element is a first preset value, it indicates that a position where the element is located should be paid attention to, and if the element value corresponding to the element is a second preset value, it indicates that the position where the element is located should not be paid attention to.

8. The method of claim 7, wherein, According to the key matrix and the real key matrix, the second key-related loss is determined, comprising: The element values corresponding to the elements in the key matrix are scaled to between the first preset value and the second preset value to obtain a scaled key matrix; According to the scaled key matrix and the real key matrix, the second key-related loss is determined.

9. The method of claim 8, wherein, According to the scaled key matrix and the real key matrix, the second key-related loss is determined, comprising: For each element in the scaled key matrix, the difference between the element value corresponding to the element and the element value corresponding to a target element is calculated, wherein the relative position of the target element in the real key matrix is the same as the relative position of the element in the key matrix; The sum of the difference values corresponding to the elements in the scaled key matrix is calculated to obtain a third sum; According to the third sum, the second key-related loss is determined.

10. The method of claim 9, wherein, According to the third sum, the second key-related loss is determined, comprising: The intersection over union of the key matrix and the real key matrix is calculated, The product of the intersection over union and the third sum is determined as the second key-related loss.

11. The method of any one of claims 1-10, wherein, After the original training sample is input into the attention model, a key matrix and a value matrix are obtained, and according to the key matrix and the value matrix, an attention loss corresponding to the original training sample is determined, comprising: After the original training sample is input into the attention model, a key matrix, a value matrix and a query matrix are obtained; Based on the key matrix and the value matrix, a first attention loss is determined; Based on the key matrix, the value matrix and the query matrix, a second attention loss is determined; Based on the first attention loss and the second attention loss, the attention loss corresponding to the original training sample is determined.

12. The method of any one of claims 1-11, wherein, The method further comprises: Obtaining a sample label corresponding to each original training sample; For each original training sample, the output corresponding to the original training sample is determined, and according to the sample label and the output corresponding to the original training sample, an output loss corresponding to the original training sample is determined; determine the first total output loss according to the output loss corresponding to each original training sample; for each target training sample, determine the output corresponding to the target training sample, and determine the output loss corresponding to the target training sample according to the sample label and the output corresponding to the target training sample; determine the second total output loss according to the output loss corresponding to each target training sample; Accordingly, according to the first total attention loss and the second total attention loss, determine whether to delete the second feature label, including: According to the first total attention loss, the second total attention loss, the first total output loss, and the second total output loss, determine whether to delete the second feature label.

13. The method of claim 12, wherein, According to the first total attention loss, the second total attention loss, the first total output loss, and the second total output loss, determine whether to delete the second feature label, including: determine the first total loss according to the first total attention loss and the first total output loss; determine the second total loss according to the second total attention loss and the second total output loss; When the difference between the second total loss and the first total loss is less than a preset threshold, determine to delete the second feature label.

14. The method of claim 13, wherein, According to the first total attention loss and the first total output loss, determine the first total loss, including: obtain the first weight corresponding to the first total attention loss and the second weight corresponding to the first total output loss; determine the first total loss according to the first total attention loss, the first total output loss, the first weight, and the second weight.

15. The method of any one of claims 12-14, wherein, The output corresponding to each original training sample is a classification probability value, which is used to indicate the probability of the original training sample being divided into each category of a plurality of preset categories, and the output loss corresponding to the original training sample is determined according to the sample label and the output corresponding to the original training sample, including: According to the following formula, the output loss corresponding to the original training sample is determined; wherein, l denotes an output loss, M denotes a total number of a plurality of preset classes, C denotes a class, P c denotes a probability of being classified into the Cth class, y c denotes a sample label, and γ is a constant greater than 1.

16. The method of any one of claims 12-15, wherein, The output corresponding to each original training sample is a classification probability value, which is used to indicate the probability of the original training sample being divided into each category of a plurality of preset categories, and the output loss corresponding to the original training sample is determined according to the sample label and the output corresponding to the original training sample, including: According to the following formula, the output loss corresponding to the original training sample is determined; wherein, l denotes an output loss, M denotes a total number of a plurality of preset classes, C denotes a class, P c denotes a probability of being classified into the Cthclass, y c denotes a sample label, and γ is a constant greater than 1.

17. A feature label screening method applied to a client, comprising: obtain a plurality of local training samples, each local training sample comprising a plurality of local feature labels and feature values corresponding to each local feature label; for each local training sample, encrypt the feature values of each local feature label corresponding to the local training sample to obtain an original training sample, wherein the encryption method is to add Laplace noise; send the plurality of original training samples to the server, each original training sample comprising a plurality of original feature labels and feature values corresponding to each original feature label; The server is configured to perform the method in any of claims 1-16. The client is configured to perform the method in any of claims 17-18. The server is configured to perform the method in any of claims 1-16.

18. The method of claim 17, wherein, The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method in any of claims 1-16. The server is configured to perform the method The acquisition module is configured to acquire a plurality of local training samples, each of the local training samples including a plurality of local feature labels and feature values corresponding to the local feature labels; The encryption module is configured to encrypt the feature values of the local feature labels corresponding to each of the local training samples to obtain original training samples, wherein the encryption manner is adding Laplace noise; The sending module is configured to send the plurality of original training samples to a server, each of the original training samples including a plurality of original feature labels and feature values corresponding to the original feature labels; The server is configured to, for each of the original training samples, obtain a key matrix and a value matrix after the original training sample is input into an attention model, and determine an attention loss corresponding to the original training sample according to the key matrix and the value matrix, wherein the attention model includes an encoder and a decoder, and the key matrix and the value matrix are transmitted from the encoder to the decoder; determine a first total attention loss according to the attention losses corresponding to the original training samples, respectively; Determine a second total attention loss based on a plurality of target training samples and the attention model, wherein the target training samples include at least one first feature label and feature values corresponding to the first feature label; The first feature label is selected from the plurality of original feature labels included in the original training samples; and determine whether to delete a second feature label according to the first total attention loss and the second total attention loss, wherein the second feature label is a feature label other than the at least one first feature label in the plurality of original feature labels.

22. An electronic device, comprising: A memory and a processor; The memory stores computer execution instructions; The processor executes the computer execution instructions stored in the memory, so that the processor executes the method of any one of claims 1-18.

23. A computer readable storage medium, the computer readable storage medium storing computer execution instructions, the computer execution instructions being executed by a processor to implement the method of any one of claims 1-18.

24. A computer program product, comprising a computer program, the computer program being executed by a processor to implement the method of any one of claims 1-18.

Images