Feature tag screening method, device and system, electronic equipment, medium and product

Abstract

The embodiment of the invention provides a feature tag screening method, device and system, electronic equipment, a medium and a product. The method comprises the following steps: acquiring a plurality of original training samples; for each original training sample, after the original training sample is input into an attention model, a key matrix and a value matrix are obtained, and attention loss corresponding to the original training sample is determined according to the key matrix and the value matrix; determining a first total attention loss according to the attention loss corresponding to each original training sample; determining a second total attention loss based on the plurality of target training samples and an attention model; the target training sample comprises at least one first feature tag; the first feature tag is screened from a plurality of original feature tags; according to the first total attention loss and the second total attention loss, whether the second feature tag is deleted is determined, the second feature tag is the feature tag except the at least one first feature tag, and the accuracy of feature tag screening can be effectively improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a method, device, system, electronic device, medium and product for screening feature labels. Background Art

[0002] With the continuous iteration and innovation of financial products and financial services, the number and types of labels are increasing. Although the increase in the number of labels has effectively expanded the user portrait features and the overall label library, the redundant labels not only increase the noise in the data, making it more difficult to extract valuable information from it, but also may cause data confusion, affecting the accuracy and reliability of the analysis results.

[0003] Therefore, how to effectively screen labels that are truly valuable needs to be solved urgently. Summary of the invention

[0004] The embodiments of the present application provide a method, device, system, electronic device, medium and product for screening feature labels to improve the accuracy of screening feature labels.

[0005] In a first aspect, an embodiment of the present application provides a method for screening feature tags, comprising:

[0006] Acquire multiple original training samples, each of which includes multiple original feature labels and feature values ​​corresponding to each original feature label;

[0007] For each original training sample, after the original training sample is input into the attention model, a key matrix and a value matrix are obtained, and an attention loss corresponding to the original training sample is determined according to the key matrix and the value matrix, wherein the attention model includes an encoder and a decoder, and the key matrix and the value matrix are transmitted by the encoder to the decoder;

[0008] Determine a first total attention loss according to the attention losses corresponding to the original training samples;

[0009] Determine a second total attention loss based on multiple target training samples and an attention model; the target training sample includes at least one first feature label and a feature value corresponding to each first feature label; the first feature label is selected from multiple original feature labels included in the original training sample;

[0010] Determine whether to delete a second feature label according to the first total attention loss and the second total attention loss, wherein the second feature label is a feature label other than at least one first feature label among multiple original feature labels.

[0011] In a possible implementation, the attention loss includes a first attention loss; and determining the attention loss corresponding to the original training sample according to the key matrix and the value matrix includes:

[0012] Obtaining a true key matrix and a true value matrix corresponding to the original training sample;

[0013] Determining a first key-related loss based on the key matrix and the true key matrix, and determining a first value-related loss based on the value matrix and the true value matrix;

[0014] Determine a first attention loss corresponding to the original training sample according to the first key-related loss and the first value-related loss.

[0015] In a possible implementation, determining a first key-related loss according to the key matrix and the true key matrix, and determining a first value-related loss according to the value matrix and the true value matrix includes:

[0016] calculating a first mutual information between the key matrix and the real key matrix, and determining a first key-related loss according to the first mutual information;

[0017] A second mutual information between the value matrix and the true value matrix is ​​calculated, and a first value correlation loss is determined according to the second mutual information.

[0018] In a possible implementation manner, determining a first key-related loss according to the first mutual information includes:

[0019] Calculating a first Pearson correlation coefficient between the bond matrix and the true bond matrix;

[0020] Adding the first mutual information and the first Pearson correlation coefficient to obtain a first sum;

[0021] determining half of the first sum as the first bond-related loss;

[0022] Correspondingly, determining the first value correlation loss according to the second mutual information includes:

[0023] Calculate the second Pearson correlation coefficient between the value matrix and the true value matrix;

[0024] Adding the second mutual information and the second Pearson correlation coefficient to obtain a second sum;

[0025] Half of the second sum is determined as the first value-related loss.

[0026] In a possible implementation, the attention loss includes a second attention loss; after the original training sample is input into the attention model, a key matrix and a value matrix are obtained, and the attention loss corresponding to the original training sample is determined according to the key matrix and the value matrix, including:

[0027] After the original training samples are input into the attention model, a key matrix, a value matrix and a query matrix are obtained;

[0028] Determine a second attention loss corresponding to the original training sample according to the key matrix, the value matrix and the query matrix.

[0029] In a possible implementation, determining the second attention loss corresponding to the original training sample according to the key matrix, the value matrix, and the query matrix includes:

[0030] Obtaining a true key matrix, a true value matrix, and a true query matrix corresponding to the original training sample;

[0031] determining a second bond-related loss based on the bond matrix and the real bond matrix;

[0032] Determine a second value-related loss based on the value matrix and the true value matrix;

[0033] Determining query-related losses according to the query matrix and the real query matrix;

[0034] A second attention loss corresponding to the original training sample is determined according to the second key-related loss, the second value-related loss and the query-related loss.

[0035] In a possible implementation, the real key matrix, the real value matrix and the real query matrix each include multiple elements. For each element, if the element value corresponding to the element is a first preset value, it means that the position of the element should be paid attention to; if the element value corresponding to the element is a second preset value, it means that the position of the element should not be paid attention to.

[0036] In a possible implementation manner, determining the second bond-related loss according to the bond matrix and the real bond matrix includes:

[0037] Scaling the element value corresponding to each element in the key matrix to between a first preset value and a second preset value to obtain a scaled key matrix;

[0038] The second bond-related loss is determined based on the scaled bond matrix and the true bond matrix.

[0039] In a possible implementation, determining the second bond-related loss according to the scaled bond matrix and the real bond matrix includes:

[0040] For each element in the scaled key matrix, calculate the difference between the element value corresponding to the element and the element value corresponding to the target element; wherein the relative position of the target element in the real key matrix is ​​the same as the relative position of the element in the key matrix;

[0041] Calculate the sum of the differences corresponding to each element in the scaled key matrix to obtain a third sum;

[0042] Based on the third sum, a second bond-related loss is determined.

[0043] In a possible implementation manner, determining the second key-related loss according to the third sum includes:

[0044] Calculate the intersection-over-union ratio of the key matrix and the real key matrix,

[0045] A product of the intersection-over-union ratio and the third sum is determined as a second bond-related loss.

[0046] In a possible implementation, after the original training sample is input into the attention model, a key matrix and a value matrix are obtained, and the attention loss corresponding to the original training sample is determined according to the key matrix and the value matrix, including:

[0047] After the original training samples are input into the attention model, a key matrix, a value matrix and a query matrix are obtained;

[0048] determining a first attention loss based on the key matrix and the value matrix;

[0049] determining a second attention loss based on the key matrix, the value matrix, and the query matrix;

[0050] Based on the first attention loss and the second attention loss, an attention loss corresponding to the original training sample is determined.

[0051] In a possible implementation, the method further includes:

[0052] Obtain the sample labels corresponding to each original training sample;

[0053] For each original training sample, determine the output corresponding to the original training sample, and determine the output loss corresponding to the original training sample according to the sample label and the output corresponding to the original training sample;

[0054] Determine a first total output loss according to the output losses corresponding to the original training samples;

[0055] For each target training sample, determine the output corresponding to the target training sample, and determine the output loss corresponding to the target training sample according to the sample label and output corresponding to the target training sample;

[0056] Determine a second total output loss according to the output losses corresponding to each target training sample;

[0057] Accordingly, determining whether to delete the second feature label according to the first total attention loss and the second total attention loss includes:

[0058] Determine whether to delete the second feature label according to the first total attention loss, the second total attention loss, the first total output loss, and the second total output loss.

[0059] In a possible implementation, determining whether to delete the second feature label according to the first total attention loss, the second total attention loss, the first total output loss, and the second total output loss includes:

[0060] Determine a first total loss according to the first total attention loss and the first total output loss;

[0061] Determining a second total loss according to the second total attention loss and the second total output loss;

[0062] When the difference between the second total loss and the first total loss is smaller than a preset threshold, it is determined to delete the second feature tag.

[0063] In a possible implementation manner, determining the first total loss according to the first total attention loss and the first total output loss includes:

[0064] Obtaining a first weight corresponding to the first total attention loss and a second weight corresponding to the first total output loss;

[0065] A first total loss is determined according to the first total attention loss, the first total output loss, the first weight and the second weight.

[0066] In a possible implementation, the output corresponding to each original training sample is a classification probability value, and the classification probability value is used to indicate the probability of the original training sample being classified into each category in a plurality of preset categories. According to the sample label and the output corresponding to the original training sample, the output loss corresponding to the original training sample is determined, including:

[0067] According to the following formula, the output loss corresponding to the original training sample is determined;

[0068]

[0069] Among them, l represents the output loss, M represents the total number of preset categories, C represents the category, P c represents the probability of being classified into the Cth class, y c represents the sample label, and γ is a constant greater than 1.

[0070] In a possible implementation, the output corresponding to each original training sample is a classification probability value, and the classification probability value is used to indicate the probability of the original training sample being classified into each category in a plurality of preset categories. According to the sample label and the output corresponding to the original training sample, the output loss corresponding to the original training sample is determined, including:

[0071] According to the following formula, the output loss corresponding to the original training sample is determined;

[0072]

[0073] Among them, l represents the output loss, M represents the total number of preset categories, C represents the category, P c represents the probability of being classified into the Cth class, y c represents the sample label, and γ is a constant greater than 1.

[0074] In a second aspect, an embodiment of the present application provides a feature tag screening method, which is applied to a client, comprising:

[0075] Acquire multiple local training samples, each local training sample including multiple local feature labels and feature values ​​corresponding to each local feature label;

[0076] For each local training sample, encrypt the feature values ​​of each local feature label corresponding to the local training sample to obtain the original training sample, wherein the encryption method is to add Laplace noise;

[0077] A plurality of original training samples are sent to a server, each of which includes a plurality of original feature labels and feature values ​​corresponding to each original feature label; so that the server obtains a key matrix and a value matrix for each original training sample after the original training sample is input into an attention model, and determines the attention loss corresponding to the original training sample according to the key matrix and the value matrix, wherein the attention model includes an encoder and a decoder, and the key matrix and the value matrix are transmitted from the encoder to the decoder; a first total attention loss is determined according to the attention losses corresponding to each original training sample; a second total attention loss is determined based on a plurality of target training samples and the attention model; the target training sample includes at least one first feature label and feature values ​​corresponding to each first feature label; the first feature label is selected from a plurality of original feature labels included in the original training sample; and whether to delete the second feature label is determined according to the first total attention loss and the second total attention loss, wherein the second feature label is a feature label other than at least one first feature label among the plurality of original feature labels.

[0078] In a possible implementation, the method further includes:

[0079] Normalize the feature values ​​corresponding to each local feature label to a preset dimension;

[0080] Correspondingly, the feature values ​​of each local feature label corresponding to the local training sample are encrypted to obtain the original training sample, including:

[0081] The normalized eigenvalues ​​corresponding to the local training samples are encrypted to obtain original training samples.

[0082] In a third aspect, an embodiment of the present application provides a feature tag screening system, including:

[0083] A server, configured to execute the method described in the first aspect;

[0084] The client is used to execute the method described in the second aspect.

[0085] In a fourth aspect, an embodiment of the present application provides a feature label screening device, comprising:

[0086] An acquisition module is used to acquire a plurality of original training samples, each of which includes a plurality of original feature labels and feature values ​​corresponding to the original feature labels;

[0087] A first determination module is used to obtain a key matrix and a value matrix for each original training sample after the original training sample is input into an attention model, and determine the attention loss corresponding to the original training sample according to the key matrix and the value matrix, wherein the attention model includes an encoder and a decoder, and the key matrix and the value matrix are transmitted by the encoder to the decoder;

[0088] A second determination module is used to determine a first total attention loss according to the attention losses corresponding to the original training samples;

[0089] A third determination module is used to determine a second total attention loss based on multiple target training samples and an attention model; the target training sample includes at least one first feature label and a feature value corresponding to each first feature label; the first feature label is selected from multiple original feature labels included in the original training sample;

[0090] The fourth determination module is used to determine whether to delete a second feature label according to the first total attention loss and the second total attention loss, wherein the second feature label is a feature label other than at least one first feature label in multiple original feature labels.

[0091] In a fourth aspect, an embodiment of the present application provides a feature tag screening device, which is applied to a client, including:

[0092] An acquisition module is used to acquire multiple local training samples, each of which includes multiple local feature labels and feature values ​​corresponding to each local feature label;

[0093] An encryption module is used to encrypt, for each local training sample, the feature values ​​of each local feature label corresponding to the local training sample to obtain the original training sample, wherein the encryption method is to add Laplace noise;

[0094] A sending module is used to send multiple original training samples to a server, each original training sample includes multiple original feature labels and feature values ​​corresponding to each original feature label; so that the server obtains a key matrix and a value matrix for each original training sample after the original training sample is input into an attention model, and determines the attention loss corresponding to the original training sample according to the key matrix and the value matrix, the attention model includes an encoder and a decoder, and the key matrix and the value matrix are transmitted by the encoder to the decoder; according to the attention losses corresponding to each original training sample, a first total attention loss is determined; based on multiple target training samples and the attention model, a second total attention loss is determined; the target training sample includes at least one first feature label and feature values ​​corresponding to each first feature label; the first feature label is screened from multiple original feature labels included in the original training sample; according to the first total attention loss and the second total attention loss, it is determined whether to delete the second feature label, wherein the second feature label is a feature label other than at least one first feature label in multiple original feature labels.

[0095] In a fifth aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor;

[0096] The memory stores computer-executable instructions;

[0097] The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method described in any one of the above aspects.

[0098] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method described in any of the above aspects.

[0099] In the seventh aspect, the embodiments of the present application provide a computer program product, including a computer program, which implements the method described in any of the above aspects when executed by a processor. The feature label screening method, device, system, electronic device, medium and product provided in the embodiments of the present application obtain multiple original training samples, each original training sample includes multiple original feature labels and feature values ​​corresponding to each original feature label; for each original training sample, after the original training sample is input into the attention model, a key matrix and a value matrix are obtained, and the attention loss corresponding to the original training sample is determined according to the key matrix and the value matrix, and the attention model includes an encoder and a decoder, and the key matrix and the value matrix are passed from the encoder to the decoder; according to the attention losses corresponding to each original training sample, a first total attention loss is determined; based on multiple target training samples and the attention model, a second total attention loss is determined. loss; the target training sample includes at least one first feature label and the feature values ​​corresponding to each first feature label; the first feature label is screened out from multiple original feature labels included in the original training sample; according to the first total attention loss and the second total attention loss, determine whether to delete the second feature label, wherein the second feature label is a feature label other than at least one first feature label in multiple original feature labels. The present application creatively uses attention loss as an indicator to measure the importance of feature labels, which can effectively improve the accuracy of feature label screening, and by screening out the most important feature labels, it can reduce the interference of noise and redundant information, so that the model can focus more on key features, and improve the generalization ability and prediction accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0100] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0101] Figure 1 An application scenario diagram provided for an embodiment of the present application;

[0102] Figure 2 A flowchart of a feature tag screening method provided in an embodiment of the present application;

[0103] Figure 3 A schematic diagram of a real key matrix provided in an embodiment of the present application;

[0104] Figure 4 A schematic diagram of the relationship between output loss and classification probability value provided in an embodiment of the present application;

[0105] Figure 5 Another feature tag screening method provided in an embodiment of the present application;

[0106] Figure 6 A schematic diagram of customer group feature label clustering and dimensionality reduction provided in an embodiment of the present application;

[0107] Figure 7 A real attention matrix provided for an embodiment of the present application;

[0108] Figure 8 A schematic diagram of difference calculation provided in an embodiment of the present application;

[0109] Fig. 9 A schematic diagram of image attention drift and IOU provided in an embodiment of the present application;

[0110] Fig.10 A schematic diagram of the structure of a feature label screening device provided in an embodiment of the present application;

[0111] Fig.11 A schematic diagram of the structure of another feature label screening device provided in an embodiment of the present application;

[0112] Fig.12 A schematic diagram of the structure of the electronic device provided in this application.

[0113] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0114] Here, exemplary embodiments will be described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are only examples of devices and methods consistent with some aspects of the present application.

[0115] With the continuous iteration and innovation of financial products and financial services, the number and types of labels are increasing. Although the increase in the number of labels has effectively expanded the user portrait features and the overall label library, the redundant labels not only increase the noise in the data, making it more difficult to extract valuable information from it, but also may cause data confusion, affecting the accuracy and reliability of the analysis results. This phenomenon is particularly prominent in big data analysis and machine learning models, because redundant labels will increase the complexity of the model, leading to overfitting problems, thereby reducing the generalization ability of the model. In addition, redundant labels will increase the cost of data storage and processing, affecting the performance and efficiency of the system.

[0116] In view of this, the present application provides a feature label screening method, which can obtain multiple original training samples, each of which includes multiple original feature labels and feature values ​​corresponding to each original feature label; for each original training sample, the original training sample is input into an attention model to obtain a key matrix and a value matrix, and the attention loss corresponding to the original training sample is determined according to the key matrix and the value matrix, the attention model includes an encoder and a decoder, and the key matrix and the value matrix are transmitted from the encoder to the decoder; according to the attention losses corresponding to each original training sample, a first total attention loss is determined; for each original training sample, at least one first feature label, at least one first feature label and the feature values ​​corresponding to each first feature label are screened out from multiple original feature labels corresponding to the original training sample to form a target training sample; multiple target training samples are respectively input into the attention model to determine a second total attention loss; wherein, the method for determining the second total attention loss is the same as the method for determining the first total attention loss, and whether to delete the second feature label is determined according to the first total attention loss and the second total attention loss, wherein the second feature label is used to indicate a feature label other than the first feature label in the original feature label. This application creatively uses attention loss as an indicator to measure the importance of feature labels, which can effectively improve the accuracy of feature label screening. By screening out the most important feature labels, it can reduce the interference of noise and redundant information, allowing the model to focus more on key features and improve the model's generalization ability and prediction accuracy.

[0117] Figure 1 An application scenario diagram provided for an embodiment of the present application, such as Figure 1 As shown, the server first obtains multiple original training samples, and for each original training sample, inputs the original training sample into the encoder in the attention model to obtain a key matrix and a value matrix. The attention model includes an encoder and a decoder. The key matrix and the value matrix are transmitted by the encoder to the decoder, and the attention loss corresponding to the original training sample is determined according to the key matrix and the value matrix; the first total attention loss is determined according to the attention losses corresponding to each original training sample; for each original training sample, at least one first feature label is selected from the multiple original feature labels corresponding to the original training sample to form a target training sample; for each target training sample, the target training sample is input into the encoder in the attention model to obtain the corresponding key matrix and value matrix, and the attention loss corresponding to the target training sample is determined according to the obtained key matrix and the value matrix, and the second total attention loss is determined according to the attention losses corresponding to each target training sample, and whether to delete the second feature label is determined according to the first total attention loss and the second total attention loss, wherein the second feature label is used to indicate the feature label other than the first feature label in the original feature label.

[0118] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0119] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0120] Figure 2 This is a flow chart of a feature tag screening method provided in an embodiment of the present application. The execution subject of this embodiment can be any device with data processing function. This application uses the server as the execution subject for specific description. Figure 2 As shown, a feature tag screening method provided in an embodiment of the present application may include:

[0121] Step 201: Acquire multiple original training samples, each of which includes multiple original feature labels and feature values ​​corresponding to each original feature label.

[0122] The feature tag screening method provided in this application can be applied to any field that requires feature tag screening, and this application does not limit this.

[0123] Exemplarily, when applied to the financial field, the original feature labels can be used to indicate the user's basic information, financial information, credit information, investment behavior, transaction behavior, account information, geographic information, behavior information, etc. For example, the original feature labels of each original training sample may include the user's age, gender, marital status, total assets, total liabilities, expected records, default records, transaction amount, transaction frequency, etc.

[0124] Exemplarily, when applied to the image field, the original feature label can be used to indicate the basic features and advanced features of the image, wherein the basic features of the image may include: the color value of each pixel (such as RGB (Red, Green, Blue) value), image size, etc., and the advanced features of the image may be extracted based on the basic features of the image, such as texture features, shape features, semantic features, frequency features, etc. For example, the original feature label of each original training sample may include: the color value of each pixel, image size, texture features, shape features, etc.

[0125] Specifically, the server may obtain a plurality of original training samples, each of which includes a plurality of original feature labels and feature values ​​corresponding to each original feature label.

[0126] Step 202: For each original training sample, after the original training sample is input into the attention model, a key matrix and a value matrix are obtained, and the attention loss corresponding to the original training sample is determined according to the key matrix and the value matrix. The attention model includes an encoder and a decoder, and the key matrix and the value matrix are passed from the encoder to the decoder.

[0127] Among them, the key matrix is ​​obtained by applying a linear transformation to the input sequence corresponding to the original training sample, which can represent the characteristics of each element in the input sequence.

[0128] The value matrix is ​​obtained by applying another linear transformation to the input sequence corresponding to the original training samples, which can represent the actual content of each element in the input sequence.

[0129] Specifically, the attention model includes an encoder and a decoder. For each original training sample, the server inputs the original training sample into the encoder in the attention model, and the encoder outputs a key matrix and a value matrix, and inputs the output key matrix and value matrix into the decoder. The server determines the corresponding attention loss of the original training sample based on the key matrix and value matrix passed from the encoder to the decoder.

[0130] Optionally, the attention loss includes a first attention loss; and determining the attention loss corresponding to the original training sample according to the key matrix and the value matrix includes:

[0131] Obtaining a true key matrix and a true value matrix corresponding to the original training sample;

[0132] Determining a first key-related loss based on the key matrix and the true key matrix, and determining a first value-related loss based on the value matrix and the true value matrix;

[0133] Determine a first attention loss corresponding to the original training sample according to the first key-related loss and the first value-related loss.

[0134] Among them, the methods for determining the real key matrix and the real value matrix are both existing technologies and will not be described in detail in this application.

[0135] The real key matrix and the real value matrix corresponding to the multiple original training samples can be stored in a preset position. For each original training sample, the server obtains the real key matrix and the real value matrix corresponding to the original training sample from the preset position, and then determines the first key-related loss according to the key matrix and the real key matrix, and determines the first value-related loss according to the value matrix and the real value matrix; finally, the first key-related loss and the first value-related loss determine the first attention loss corresponding to the original training sample.

[0136] In this way, the first key-related loss and the first value-related loss respectively measure the differences between the key matrix and the value matrix and their true values. According to the first key-related loss and the first value-related loss, the first attention loss corresponding to the original training sample is determined, which can improve the accuracy of the determined first attention loss.

[0137] Optionally, determining a first key-related loss according to the key matrix and the true key matrix, and determining a first value-related loss according to the value matrix and the true value matrix includes:

[0138] calculating a first mutual information between the key matrix and the real key matrix, and determining a first key-related loss according to the first mutual information;

[0139] A second mutual information between the value matrix and the true value matrix is ​​calculated, and a first value correlation loss is determined according to the second mutual information.

[0140] Among them, mutual information is an important concept in information theory, which is used to measure the degree of dependence or information sharing between two random variables. Specifically, mutual information quantifies the amount of information one random variable contains about another random variable.

[0141] For example, the bond matrix is ​​K f The real bond matrix is ​​represented by K g It means that I(K g ; K f ) is the mutual information between the key matrix and the true key matrix, which measures K g K f degree.

[0142] The value matrix is ​​V f Indicated by V, the real value matrix g It means that I(V g ; V f ) is the mutual information between the value matrix and the true value matrix, which measures V g Contains V f degree.

[0143] Specifically, the first mutual information between the key matrix and the true key matrix is ​​calculated, and the first key-related loss is determined according to the first mutual information; the second mutual information between the value matrix and the true value matrix is ​​calculated, and the second key-related loss is determined according to the second mutual information.

[0144] In this way, the mutual information quantifies the dependency and information sharing between the two matrices, thereby improving the accuracy of determining the first-key correlation loss and the first-value correlation loss.

[0145] Optionally, determining a first key-related loss according to the first mutual information includes:

[0146] Calculating a first Pearson correlation coefficient between the bond matrix and the true bond matrix;

[0147] Adding the first mutual information and the first Pearson correlation coefficient to obtain a first sum;

[0148] determining half of the first sum as the first bond-related loss;

[0149] Correspondingly, determining the first value correlation loss according to the second mutual information includes:

[0150] Calculate the second Pearson correlation coefficient between the value matrix and the true value matrix;

[0151] Adding the second mutual information and the second Pearson correlation coefficient to obtain a second sum;

[0152] Half of the second sum is determined as the first value-related loss.

[0153] Among them, the Pearson correlation coefficient, also known as the Pearson product-moment correlation coefficient, is a statistic used to measure the degree of linear correlation between two variables.

[0154] Specifically, is the Pearson correlation coefficient between the bond matrix and the true bond matrix, which is used to measure the linear correlation between the bond matrix and the true bond matrix. is the Pearson correlation coefficient between the value matrix and the true value matrix, which is used to measure the linear correlation between the value matrix and the true value matrix.

[0155] Specifically, the server calculates the first Pearson correlation coefficient between the key matrix and the true key matrix The first mutual information I(K g ; K f ) and the first Pearson correlation coefficient Add them together to get the first sum: The first key correlation loss is

[0156] Similarly, the server calculates the second Pearson correlation coefficient between the value matrix and the true value matrix The second mutual information I(V g ; V f ) and the second Pearson correlation coefficient Add them together to get the second sum: The first value of the associated loss is

[0157] In this way, mutual information and Pearson correlation coefficient measure the dependency between matrices from different perspectives. By combining these two measurement methods, the accuracy of the determined first key correlation loss and first value correlation loss can be further improved.

[0158] Optionally, the attention loss includes a second attention loss; after the original training sample is input into the attention model, a key matrix and a value matrix are obtained, and the attention loss corresponding to the original training sample is determined according to the key matrix and the value matrix, including:

[0159] After the original training samples are input into the attention model, a key matrix, a value matrix and a query matrix are obtained;

[0160] Determine a second attention loss corresponding to the original training sample according to the key matrix, the value matrix and the query matrix.

[0161] Among them, the query matrix is ​​obtained by linearly transforming the input sequence corresponding to the original training sample, which can represent the input features of the current step of the decoder and is used to calculate the attention weight with the key matrix.

[0162] Specifically, for each original training sample, the server inputs the original training sample into the encoder of the attention model, and the encoder outputs a key matrix, a value matrix, and a query matrix. The attention loss corresponding to the original training sample is determined based on the key matrix, value matrix, and query matrix output by the encoder.

[0163] In this way, by determining the attention loss based on the key matrix, the value matrix and the query matrix, the accuracy of the determined attention loss can be further improved.

[0164] Optionally, determining a second attention loss corresponding to the original training sample according to the key matrix, the value matrix, and the query matrix includes:

[0165] Obtaining a true key matrix, a true value matrix, and a true query matrix corresponding to the original training sample;

[0166] determining a second bond-related loss based on the bond matrix and the real bond matrix;

[0167] Determine a second value-related loss based on the value matrix and the true value matrix;

[0168] Determining query-related losses according to the query matrix and the real query matrix;

[0169] A second attention loss corresponding to the original training sample is determined according to the second key-related loss, the second value-related loss and the query-related loss.

[0170] Among them, the method of determining the real key matrix, the real value matrix and the real query matrix is ​​a prior art and will not be described in detail in this application.

[0171] Specifically, the server determines the second key-related loss based on the key matrix and the true key matrix; determines the second value-related loss based on the value matrix and the true value matrix; determines the query-related loss based on the query matrix and the true query matrix; and determines the second attention loss corresponding to the original training sample based on the second key-related loss, the second value-related loss and the query-related loss.

[0172] In the present application, the methods for calculating the second key related loss, the second value related loss and the query related loss are all the same.

[0173] In this way, calculating the key-related loss, value-related loss, and query-related loss separately can more comprehensively measure the difference between the matrix generated by the model and the true matrix, thereby improving the accuracy of the determined second attention loss.

[0174] Optionally, the real key matrix, real value matrix and real query matrix each include multiple elements. For each element, if the element value corresponding to the element is a first preset value, it means that the position of the element should be paid attention to; if the element value corresponding to the element is a second preset value, it means that the position of the element should not be paid attention to.

[0175] For example, in the image field, the area of ​​an image that needs to be noticed is a square of 4*4 (pixels) in the center of the image, the first preset value is 1, the second preset value is 0, Figure 3 A schematic diagram of a real key matrix provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, the element values ​​corresponding to the elements in the 4*4 area in the center of the matrix are all 1, and the element values ​​of the remaining elements are all 0. For any element, if the element value corresponding to the element is 1, it means that the position of the element should be paid attention to. If the element value corresponding to the element is 0, it means that the position of the element should not be paid attention to.

[0176] The true value matrix and the true query matrix can be the same as the true key matrix.

[0177] In this way, by using preset values ​​to mark the positions that should be paid attention to and those that should not be paid attention to, the real distribution of attention can be more accurately reflected, which is beneficial to subsequent numerical calculations and improves computational efficiency.

[0178] Optionally, determining a second bond-related loss according to the bond matrix and the real bond matrix includes:

[0179] Scaling the element value corresponding to each element in the key matrix to between a first preset value and a second preset value to obtain a scaled key matrix;

[0180] The second bond-related loss is determined based on the scaled bond matrix and the true bond matrix.

[0181] Specifically, the server scales the element value corresponding to each element in the key matrix to between a first preset value and a second preset value, such as between 0 and 1, to obtain a scaled key matrix; in the scaled key matrix, the element value corresponding to each element is between the first preset value and the second preset value, and the second key-related loss is determined based on the scaled key matrix and the real key matrix.

[0182] In this way, scaling the element values ​​of the key matrix to between the first preset value and the second preset value helps to avoid numerical overflow or underflow and enhance the numerical stability during the calculation process. Determining the second key-related loss based on the scaled key matrix and the real key matrix can improve the accuracy of the determined second key-related loss.

[0183] Optionally, determining a second bond-related loss according to the scaled bond matrix and the real bond matrix includes:

[0184] For each element in the scaled key matrix, calculate the difference between the element value corresponding to the element and the element value corresponding to the target element; wherein the relative position of the target element in the real key matrix is ​​the same as the relative position of the element in the key matrix;

[0185] Calculate the sum of the differences corresponding to each element in the scaled key matrix to obtain a third sum;

[0186] Based on the third sum, a second bond-related loss is determined.

[0187] Exemplarily, the scaled key matrix and the real key matrix are both 3*3 matrices. For the element in the first row and first column of the scaled key matrix, the difference d1 between the element and the target element in the first row and first column of the real key matrix is ​​calculated. For the element in the first row and second column of the scaled key matrix, the difference d1 between the element and the target element in the first row and second column of the real key matrix is ​​calculated… and so on. For the element in the third row and third column of the scaled key matrix, the difference d9 between the element and the target element in the third row and third column of the real key matrix is ​​calculated. The third sum is equal to d1+d2…+d9. The second key-related loss is determined based on the third sum.

[0188] In this way, by calculating the difference between the scaled key matrix and the true key matrix element by element, the error of each element can be accurately captured, thereby improving the accuracy of the loss calculation, and the difference is accumulated element by element into the third sum, which simplifies the loss calculation process and makes the calculation of the loss function more intuitive and easy to implement.

[0189] Optionally, determining a second key-related loss according to the third sum comprises:

[0190] Calculate the intersection-over-union ratio of the key matrix and the real key matrix,

[0191] A product of the intersection-over-union ratio and the third sum is determined as a second bond-related loss.

[0192] Among them, calculating the intersection over union (IOU) of two matrices is usually used to evaluate the degree of overlap between the two matrices.

[0193] Specifically, the server calculates the intersection-and-union ratio of the key matrix to be the real key matrix, and determines the product of the intersection-and-union ratio and the third sum as the second key-related loss.

[0194] In this way, by calculating the intersection-over-union ratio of the key matrix and the true key matrix, the overlap between the two matrices can be evaluated. Combining the intersection-over-union ratio with the third sum can more comprehensively evaluate the difference between the key matrix and the true key matrix, thereby obtaining a more accurate loss value.

[0195] Optionally, after the original training sample is input into the attention model, a key matrix and a value matrix are obtained, and the attention loss corresponding to the original training sample is determined according to the key matrix and the value matrix, including:

[0196] After the original training samples are input into the attention model, a key matrix, a value matrix and a query matrix are obtained;

[0197] determining a first attention loss based on the key matrix and the value matrix;

[0198] determining a second attention loss based on the key matrix, the value matrix, and the query matrix;

[0199] Based on the first attention loss and the second attention loss, an attention loss corresponding to the original training sample is determined.

[0200] Specifically, the server inputs the original training samples into the encoder of the attention model to obtain a key matrix, a value matrix and a query matrix; based on the key matrix and the value matrix, the first attention loss can be determined, and the specific determination method can refer to any of the above embodiments, which will not be repeated here; based on the key matrix, the value matrix and the query matrix, the second attention loss can be determined, and the specific determination method can also refer to any of the above embodiments, which will not be repeated here.

[0201] Based on the first attention loss and the second attention loss, the attention loss corresponding to the original training sample can be determined in a variety of ways, which is not limited in this application.

[0202] In an optional implementation, the first attention loss and the second attention loss are added to obtain the attention loss corresponding to the original training sample.

[0203] In another optional implementation, the server first obtains the weight α corresponding to the first attention loss and the weight β corresponding to the second attention loss, and the attention loss corresponding to the original training sample = first attention loss × α + second attention loss × β.

[0204] In this way, by respectively calculating the first attention loss based on the key matrix and the value matrix and the second attention loss based on the key matrix, the value matrix and the query matrix, the attention loss can be evaluated from multiple dimensions, thereby improving the accuracy of the attention loss corresponding to the determined original training samples.

[0205] Step 203: Determine a first total attention loss according to the attention losses corresponding to the original training samples.

[0206] Add up the attention losses corresponding to each original training sample, and the sum is the first total attention loss.

[0207] Step 204: determine a second total attention loss based on multiple target training samples and an attention model; the target training samples include at least one first feature label and a feature value corresponding to each first feature label; the first feature label is selected from multiple original feature labels included in the original training samples.

[0208] Specifically, for each original training sample, the server selects at least one first feature label, at least one first feature label and feature values ​​corresponding to each first feature label from multiple original feature labels corresponding to the original training sample to form a target training sample. The screening process can be random screening or screening according to preset rules.

[0209] The server inputs the multiple target training samples into the attention model respectively to determine the second total attention loss; wherein the method for determining the second total attention loss is the same as the method for determining the first total attention loss, which is as follows:

[0210] For each target training sample, the target training sample is input into the encoder in the attention model to obtain the corresponding key matrix and value matrix, and the attention loss corresponding to the target training sample is determined based on the obtained key matrix and setting matrix, wherein the method of determining the attention loss corresponding to the target training sample based on the obtained key matrix and setting matrix is ​​the same as the method of determining the attention loss corresponding to the original sample based on the key matrix and setting matrix, and reference may be made to the method in any of the above embodiments.

[0211] The server adds up the attention losses corresponding to each target training sample, and the obtained sum is the second total attention loss.

[0212] Step 205: Determine whether to delete a second feature tag based on the first total attention loss and the second total attention loss, wherein the second feature tag is a feature tag among multiple original feature tags except for at least one first feature tag.

[0213] Specifically, when the difference between the first total attention loss and the second total attention loss is less than a preset threshold, it is determined to delete the second feature label.

[0214] When the difference between the first total attention loss and the second total attention loss is greater than a preset threshold, it is determined that the second feature label is not deleted. Further, at least one third feature label is selected from the second feature label, and for each original training sample, the third feature label corresponding to the original sample is combined with the first feature label to obtain a first target training sample, and the first target training sample is input into the attention model to determine the corresponding third attention loss. When the difference between the third attention loss and the first total attention loss is less than a preset threshold, the fourth feature label other than the third feature label in the second feature label is deleted. When the difference between the third attention loss and the first total attention loss is greater than a preset threshold, the fourth feature label is not deleted.

[0215] The above method can be used to determine whether each original feature label is deleted.

[0216] Exemplarily, the original feature labels corresponding to the original training samples include: user's age, gender, marital status, total assets, total liabilities, expected records, default records, transaction amount, and transaction frequency.

[0217] The first feature tags selected include: the user's age, gender, marital status, total assets, and total liabilities. The first feature tags and the feature values ​​corresponding to each first feature tag constitute the target training sample.

[0218] A first total attention loss is calculated according to each original training sample, and a second total attention loss is calculated according to each target training sample. When the difference between the first total attention loss and the second total attention loss is less than a preset difference, it is determined to delete the second feature label, that is, to delete the labels: expected records, default records, transaction amounts, and transaction frequencies.

[0219] When the difference between the first total attention loss and the second total attention loss is greater than a preset difference, it is determined not to delete the second feature tag, and a third feature tag, namely, expected record and default record, is screened out from the second feature tag.

[0220] The third feature label is combined with the first feature label to obtain a first target training sample. The labels corresponding to the first target training sample are: user's age, gender, marital status, total assets, total liabilities, expected records, and default records.

[0221] The third total attention loss is calculated based on the first target training sample. When the difference between the first total attention loss and the third total attention loss is less than the preset difference, it is determined to delete the label transaction amount and transaction frequency. When the difference between the first total attention loss and the third total attention loss is greater than the preset difference, it is determined not to delete the label transaction amount and transaction frequency.

[0222] The feature label screening method provided in the embodiment of the present application obtains multiple original training samples, each of which includes multiple original feature labels and feature values ​​corresponding to each original feature label; for each original training sample, after the original training sample is input into the attention model, a key matrix and a value matrix are obtained, and the attention loss corresponding to the original training sample is determined according to the key matrix and the value matrix, the attention model includes an encoder and a decoder, and the key matrix and the value matrix are transmitted by the encoder to the decoder; according to the attention losses corresponding to the original training samples, a first total attention loss is determined; based on multiple target training samples and the attention model, a second total attention loss is determined; the target training sample This invention includes at least one first feature label and the feature value corresponding to each first feature label; the first feature label is screened out from multiple original feature labels included in the original training sample; according to the first total attention loss and the second total attention loss, it is determined whether to delete the second feature label, wherein the second feature label is a feature label other than at least one first feature label in the multiple original feature labels. This application creatively uses attention loss as an indicator to measure the importance of feature labels, which can effectively improve the accuracy of feature label screening, and by screening out the most important feature labels, it can reduce the interference of noise and redundant information, so that the model can focus more on key features, and improve the generalization ability and prediction accuracy of the model.

[0223] Optionally, the feature tag screening method provided in the embodiment of the present application further includes:

[0224] Obtain the sample labels corresponding to each original training sample;

[0225] For each original training sample, determine the output corresponding to the original training sample, and determine the output loss corresponding to the original training sample according to the sample label and the output corresponding to the original training sample;

[0226] Determine a first total output loss according to the output losses corresponding to the original training samples;

[0227] For each target training sample, determine the output corresponding to the target training sample, and determine the output loss corresponding to the target training sample according to the sample label and output corresponding to the target training sample;

[0228] Determine a second total output loss according to the output losses corresponding to each target training sample;

[0229] Accordingly, determining whether to delete the second feature label according to the first total attention loss and the second total attention loss includes:

[0230] Determine whether to delete the second feature label according to the first total attention loss, the second total attention loss, the first total output loss, and the second total output loss.

[0231] Among them, the sample labels of the target training samples and the corresponding original training samples are the same.

[0232] Specifically, the server obtains the sample label corresponding to each original training sample; for each original training sample, the original training sample is input into the attention model, and the attention model outputs the output corresponding to the original training sample. The server determines the output loss corresponding to the original training sample based on the sample label and output corresponding to the original training sample.

[0233] Sum the output losses corresponding to the original training samples, and determine the sum as the first total output loss;

[0234] For each target training sample, the server inputs the target training sample into the attention model, and the attention model outputs the output corresponding to the target training sample. The server determines the output loss corresponding to the target training sample based on the sample label and output corresponding to the target training sample.

[0235] Sum the output losses corresponding to each target training sample, and determine the sum as the second total output loss;

[0236] Finally, the server determines whether to delete the second feature tag based on the first total attention loss, the second total attention loss, the first total output loss, and the second total output loss.

[0237] In this way, by calculating the output loss and attention loss of the original training samples and the target training samples, the importance of the feature labels can be comprehensively evaluated from more perspectives, thereby improving the accuracy of determining whether to delete the second feature label.

[0238] Optionally, determining whether to delete the second feature label according to the first total attention loss, the second total attention loss, the first total output loss, and the second total output loss includes:

[0239] Determine a first total loss according to the first total attention loss and the first total output loss;

[0240] Determining a second total loss according to the second total attention loss and the second total output loss;

[0241] When the difference between the second total loss and the first total loss is smaller than a preset threshold, it is determined to delete the second feature tag.

[0242] Specifically, the server calculates the sum of the first total attention loss and the first total output loss, and determines that the result of the sum is the first total loss; calculates the sum of the second total attention loss and the second total output loss, and determines that the result of the sum is the second total loss; when the difference between the second total loss and the first total loss is less than a preset threshold, determines to delete the second feature label, and when the difference between the second total loss and the first total loss is greater than the preset threshold, determines not to delete the second feature label.

[0243] In this way, by combining attention loss and output loss, the importance of feature labels can be evaluated more accurately, and a preset threshold can be used to automatically decide whether to delete feature labels, reducing the need for human intervention and making the feature selection process more automated and efficient.

[0244] Optionally, determining a first total loss according to the first total attention loss and the first total output loss includes:

[0245] Obtaining a first weight corresponding to the first total attention loss and a second weight corresponding to the first total output loss;

[0246] A first total loss is determined according to the first total attention loss, the first total output loss, the first weight and the second weight.

[0247] Specifically, the server obtains a first weight a corresponding to the first total attention loss and a second weight b corresponding to the first total output loss, and the first total loss = first total attention loss × a + first total output loss × b.

[0248] In this way, the introduction of weight parameters (first weight and second weight) allows weighted adjustment of different types of losses. In this way, the contribution ratio of attention loss and output loss in the total loss can be flexibly adjusted according to the needs of specific tasks, thereby improving the accuracy of determining the first total loss.

[0249] Optionally, determining a second total loss according to the second total attention loss and the second total output loss includes:

[0250] Obtain a third weight corresponding to the second total attention loss and a fourth weight corresponding to the second total output loss;

[0251] A second total loss is determined according to the second total attention loss, the second total output loss, the third weight and the fourth weight.

[0252] Optionally, the output corresponding to each original training sample is a classification probability value, and the classification probability value is used to indicate the probability of the original training sample being classified into each category among multiple preset categories. According to the sample label and output corresponding to the original training sample, the output loss corresponding to the original training sample is determined, including:

[0253] According to the following formula, the output loss corresponding to the original training sample is determined;

[0254]

[0255] Among them, l represents the output loss, M represents the total number of preset categories, C represents the category, P c represents the probability of being classified into the Cth class, y c represents the sample label, and γ is a constant greater than 1.

[0256] Specifically, the exponent γ is the loss shrinkage factor, which is used to obtain a more accurate output loss under the same conditions. Figure 4 A schematic diagram of the relationship between output loss and classification probability value provided in an embodiment of the present application is shown in FIG. Figure 4 As shown in Figure 1, the existence of γ is used to "tighten" the loss function, and as γ increases, under the same independent variable conditions, the loss value becomes smaller, that is, small values ​​are more conducive to numerical operations than large values.

[0257] In this way, by adding the loss shrinkage factor, the accuracy of the determined output loss can be improved.

[0258] Optionally, the output corresponding to each original training sample is a classification probability value, and the classification probability value is used to indicate the probability of the original training sample being classified into each category among multiple preset categories. According to the sample label and output corresponding to the original training sample, the output loss corresponding to the original training sample is determined, including:

[0259] According to the following formula, the output loss corresponding to the original training sample is determined;

[0260]

[0261] Among them, l represents the output loss, M represents the total number of preset categories, C represents the category, P c represents the probability of being classified into the Cth class, y c represents the sample label, and γ is a constant greater than 1.

[0262] In this way, for Based on the exponential γ, P is added c When the probability corresponding to a certain category is large, the larger the overall index term is, the smaller the loss value is, that is, "the greater the possibility of being correct, the smaller the loss is", because the accuracy of the determined output loss can be further improved.

[0263] Figure 5 Another feature tag screening method provided in the embodiment of the present application. The execution subject of this embodiment can be any device with data processing function, and this application is specifically described with the client as the execution subject. Figure 5 As shown, another feature tag screening method provided by an embodiment of the present application may include:

[0264] Step 501: Acquire multiple local training samples, each of which includes multiple local feature labels and feature values ​​corresponding to each local feature label.

[0265] Specifically, the client obtains a plurality of local training samples, each of which includes a plurality of local feature tags and feature values ​​corresponding to each local feature tag.

[0266] Step 502: For each local training sample, encrypt the feature values ​​of each local feature label corresponding to the local training sample to obtain the original training sample, wherein the encryption method is to add Laplace noise.

[0267] Specifically, for each local training sample, the client encrypts the feature values ​​of each local feature label corresponding to the local training sample to obtain the original training sample, wherein the encryption method is to add Laplace noise.

[0268] Step 503, sending multiple original training samples to the server, each original training sample includes multiple original feature labels and feature values ​​corresponding to each original feature label; so that the server obtains a key matrix and a value matrix for each original training sample after the original training sample is input into the attention model, and determines the attention loss corresponding to the original training sample according to the key matrix and the value matrix, the attention model includes an encoder and a decoder, and the key matrix and the value matrix are transmitted by the encoder to the decoder; according to the attention losses corresponding to each original training sample, a first total attention loss is determined; based on multiple target training samples and the attention model, a second total attention loss is determined; the target training sample includes at least one first feature label and feature values ​​corresponding to each first feature label; the first feature label is selected from the multiple original feature labels included in the original training sample; according to the first total attention loss and the second total attention loss, it is determined whether to delete the second feature label, wherein the second feature label is a feature label other than at least one first feature label in the multiple original feature labels.

[0269] Specifically, the client sends multiple original training samples to the server. After receiving the multiple original training samples, the server inputs each original training sample into the attention model to obtain a key matrix and a value matrix, and determines the attention loss corresponding to the original training sample based on the key matrix and the value matrix. The attention model includes an encoder and a decoder, and the key matrix and the value matrix are transmitted from the encoder to the decoder; a first total attention loss is determined based on the attention losses corresponding to each original training sample; a second total attention loss is determined based on multiple target training samples and the attention model; the target training sample includes at least one first feature label and feature values ​​corresponding to each first feature label; the first feature label is selected from multiple original feature labels included in the original training sample; and whether to delete the second feature label is determined based on the first total attention loss and the second total attention loss, wherein the second feature label is a feature label other than at least one first feature label in the multiple original feature labels.

[0270] In this way, the client encrypts the feature values ​​of each local feature label corresponding to the local training sample, obtains the original training sample, and then sends it to the server, which can improve the security of the data and reduce the risk of leakage of important information. In addition, this application creatively uses attention loss as an indicator to measure the importance of feature labels, which can effectively improve the accuracy of feature label screening, and by screening out the most important feature labels, it can reduce the interference of noise and redundant information, so that the model can focus more on key features, and improve the generalization ability and prediction accuracy of the model.

[0271] Optionally, the method further includes:

[0272] Normalize the feature values ​​corresponding to each local feature label to a preset dimension;

[0273] Correspondingly, the feature values ​​of each local feature label corresponding to the local training sample are encrypted to obtain the original training sample, including:

[0274] The normalized eigenvalues ​​corresponding to the local training samples are encrypted to obtain original training samples.

[0275] Specifically, the client normalizes the feature values ​​corresponding to each local feature label to a preset dimension, and then encrypts each normalized feature value to obtain an original training sample.

[0276] In this way, normalizing the eigenvalues ​​corresponding to each local feature label to a preset dimension helps to eliminate the dimensional differences between different features, allowing the eigenvalues ​​to be compared and processed on the same scale, thus avoiding the adverse effects caused by some eigenvalues ​​being too large or too small.

[0277] The present application also provides another feature tag screening method. The specific steps are as follows:

[0278] Step 1: Based on the vertical federated learning scenario, each institution (i.e., multiple clients) aligns with a unified customer base.

[0279] Step 2: Each institution uses unsupervised clustering and PCA dimensionality reduction based on its initial feature labels to filter out redundant feature labels and redundant samples. Figure 6 A schematic diagram of customer group feature label clustering and dimensionality reduction provided in an embodiment of the present application, such as Figure 6 As shown, after feature clustering and dimensionality reduction, the feature dimension is 3 (feature labels include: age, card type and maximum single consumption amount in the past 6 months), and the customer groups can be roughly divided.

[0280] Step 3: For the initial feature labels of the institutions selected in the previous step, normalize the feature values ​​corresponding to the initial feature labels to a specific dimension to avoid large numerical features (such as large transaction flow features) from appearing in subsequent calculations.

[0281] Step 4: For each institution, the normalized feature values ​​corresponding to the institution are desensitized and contributed locally (sent to the server) in the form of numerical vectors. The normalized feature desensitization matrices of multiple institutions constitute the input features of the large model.

[0282] For example, Figure 6 We normalize the feature values ​​corresponding to the three feature labels of age, the maximum single consumption amount in the past six months, and the card type to the range of 0-5. Let d = 10, then: age group / d, the maximum single consumption amount in the past six months / d, and the card type is marked as: 0-no credit card, 1-card type 1, 2-card type 2, 3-card type 3. For customer A who is 25 years old, has a maximum single consumption amount of 50,000 yuan in the past six months, and holds a card type 1, the normalized plaintext feature vector of these three feature vectors in Bank A is F A o=[2.5,5,1], the encryption method is to add Laplace noise, then the encrypted ciphertext feature vector is recorded as F A e=[lap(2.5),lap(5),lap(1)]. Similarly, assuming that the largest single purchase amount of customer A in Bank B in the past six months is 70,000 yuan and he does not have a credit card, then the plaintext vector is F B o=[2.5,7,0],the ciphertext vector is F B e=[lap(2.5),lap(7),lap(0)]. When Bank A and Bank B jointly train a large financial model in a vertical federation scenario, the desensitized feature F A e and F B e is the out-of-domain features contributed by the two banks.

[0283] Step 5: After the server receives the numerical vectors sent by each organization, it performs federated feature fusion and repeats the unsupervised clustering -> dimensionality reduction in step 2 before inputting the large model.

[0284] Step 6: For the K and V attention matrices passed from the encoder to the decoder of the large model, calculate the "attention drift loss";

[0285] The specific "attention drift loss" function is designed as follows:

[0286] (1) Exponential cross entropy loss: The cross entropy loss at the output of the large model decoder is used as the overall detection indicator of the model. It directly reflects the impact of input features on output from the perspective of output accuracy. We denote it as:

[0287]

[0288] Among them, the γ index is the loss shrinkage factor, which is used to obtain more accurate losses under the same conditions, (1–P ic ) x The existence of the exponent x of the term is used to "tighten" the loss function, and the larger the x, the smaller the loss value under the same independent variable conditions, that is, small values ​​are more conducive to numerical operations than large values. Furthermore, P is added to the exponent x. ic Term, P is the classification probability value when sample i is of category C. When the probability of a certain category is large, the larger the overall exponential term, the smaller the loss value, that is, "the greater the possibility of being correct, the smaller the loss".

[0289] (2) Attention matrix loss: In the attention model (which can be a transformer model), the output of the encoder stack is K (key matrix) and V (value matrix) in the attention matrix. K and V serve as the input of each decoder in the decoder stack, so the loss function here measures the ability of all encoders to capture the original input. Let the real attention matrix be A g , the drifting attention matrix is ​​A f (key matrix or value matrix output by the encoder), the attention matrix loss is:

[0290]

[0291] Among them, I(A g ; A f ) is the mutual information measure between the drifted attention matrix and the true attention matrix, which measures the g A f The degree of information. is the Pearson correlation coefficient between the drifted attention matrix and the Ground Truth (real) attention matrix, which is used to measure A g and A f The linear correlation between .

[0292] The above formulas are used to calculate the key-related attention loss between the key matrix and the true key matrix, and the value-related attention loss between the value matrix and the true value matrix. The calculated key-related attention loss and value-related attention loss are added together to obtain the overall attention loss of the key matrix and the value matrix.

[0293] (3) Groud Truth Masked IOU True Value Mask Loss: Borrowing the traditional "mask" idea, first, this proposal proposes a true value masked attention matrix, which represents the position where the true attention should exist in the matrix. Figure 7 A real attention matrix is ​​provided in the embodiment of the present application, such as Figure 7 As shown, the area where the elements that should be noticed constitutes the head of the kitten.

[0294] Secondly, during the model training process, we calculate the Q, K, and V matrices for each position Q i,j , K i,j 、V i,j First scale it (scale to between 0 and 1), and then compare it with each position Q of the true value mask attention matrix gi,j , K gi,j 、V gi,j Calculate the difference and write the attention matrix of Q, K, and V as A i,j ,A gi,j ), Figure 8 A difference calculation schematic diagram provided in the embodiment of the present application, such as Figure 8 As shown, the element Q in the fourth row and fifth column of the query matrix 4,5 = 0.7 and the real query matrix Q g4,5 =1 is 0.3.

[0295] Next, calculate the IOU value of the true value mask attention matrix and the Q, K, and V matrices during model training, denoted as I q ,I k ,I v (The Q, K, V attention matrices are uniformly denoted as I A ). IOU≈0.5.

[0296] Fig. 9 A schematic diagram of image attention drift and IOU provided in an embodiment of the present application is shown in FIG. Fig. 9As shown, the area within the blue border is the attention area determined by the model, the area within the green border is the real attention area, the area within the red border is the intersection area of ​​the above two attention areas, and the area within the black border is the union area of ​​the above two attention areas.

[0297] Based on the above description of the true value mask loss, the loss expression is designed as:

[0298] l1=I A ·∑|A i,j -A gi,j |

[0299] The overall model training loss function is:

[0300]

[0301] Among them, α, β, and γ are balance weight coefficients, and s is the scaling factor (because when calculating the l3 loss, it is scaled to between 0 and 1 to calculate the difference and compare the difference with the real attention matrix. Now returning to the overall loss calculation, the numerical value needs to be consistent with the other two scales).

[0302] Step 7: Feedback the "attention drift loss" to the input feature engineering to determine whether the feature labels of the large model input are deleted.

[0303] The present application also provides a feature tag screening system, the system comprising:

[0304] A server, configured to execute the method described in any of the above embodiments;

[0305] The client is used to execute the method described in any of the above embodiments.

[0306] The specific implementation principle and effect of the feature label screening system provided in the embodiment of the present application can be found in the aforementioned embodiment and will not be repeated here.

[0307] Corresponding to the above-mentioned feature label screening method, the embodiment of the present application further provides a feature label screening device, Fig.10 A structural diagram of a feature label screening device provided in an embodiment of the present application. Fig.10 As shown, the device comprises:

[0308] The acquisition module 1001 is used to acquire a plurality of original training samples, each of which includes a plurality of original feature labels and feature values ​​corresponding to the original feature labels;

[0309] A first determination module 1002 is used to obtain a key matrix and a value matrix for each original training sample after the original training sample is input into an attention model, and determine the attention loss corresponding to the original training sample according to the key matrix and the value matrix, wherein the attention model includes an encoder and a decoder, and the key matrix and the value matrix are transmitted from the encoder to the decoder;

[0310] A second determination module 1003 is used to determine a first total attention loss according to the attention losses corresponding to the original training samples;

[0311] The third determination module 1004 is used to determine a second total attention loss based on multiple target training samples and an attention model; the target training sample includes at least one first feature label and a feature value corresponding to each first feature label; the first feature label is selected from multiple original feature labels included in the original training sample;

[0312] The fourth determination module 1005 is used to determine whether to delete a second feature label according to the first total attention loss and the second total attention loss, wherein the second feature label is a feature label other than at least one first feature label in multiple original feature labels.

[0313] Optionally, the attention loss includes a first attention loss; when the first determination module 1002 determines the attention loss corresponding to the original training sample according to the key matrix and the value matrix, it is specifically used to:

[0314] Obtaining a true key matrix and a true value matrix corresponding to the original training sample;

[0315] Determining a first key-related loss based on the key matrix and the true key matrix, and determining a first value-related loss based on the value matrix and the true value matrix;

[0316] Determine a first attention loss corresponding to the original training sample according to the first key-related loss and the first value-related loss.

[0317] Optionally, when the first determining module 1002 determines the first key-related loss according to the key matrix and the true key matrix, and determines the first value-related loss according to the value matrix and the true value matrix, it is specifically used to:

[0318] calculating a first mutual information between the key matrix and the real key matrix, and determining a first key-related loss according to the first mutual information;

[0319] A second mutual information between the value matrix and the true value matrix is ​​calculated, and a first value correlation loss is determined according to the second mutual information.

[0320] Optionally, when the first determining module 1002 determines the first key-related loss according to the first mutual information, it is specifically configured to:

[0321] Calculating a first Pearson correlation coefficient between the bond matrix and the true bond matrix;

[0322] Adding the first mutual information and the first Pearson correlation coefficient to obtain a first sum;

[0323] determining half of the first sum as the first bond-related loss;

[0324] Correspondingly, determining the first value correlation loss according to the second mutual information includes:

[0325] Calculate the second Pearson correlation coefficient between the value matrix and the true value matrix;

[0326] Adding the second mutual information and the second Pearson correlation coefficient to obtain a second sum;

[0327] Half of the second sum is determined as the first value-related loss.

[0328] Optionally, the attention loss includes a second attention loss; the first determining module 1002 is specifically configured to:

[0329] After the original training samples are input into the attention model, a key matrix, a value matrix and a query matrix are obtained;

[0330] Determine a second attention loss corresponding to the original training sample according to the key matrix, the value matrix and the query matrix.

[0331] Optionally, when the first determination module 1002 determines the second attention loss corresponding to the original training sample according to the key matrix, the value matrix and the query matrix, it is specifically used to:

[0332] Obtaining a true key matrix, a true value matrix, and a true query matrix corresponding to the original training sample;

[0333] determining a second bond-related loss based on the bond matrix and the real bond matrix;

[0334] Determine a second value-related loss based on the value matrix and the true value matrix;

[0335] Determining query-related losses according to the query matrix and the real query matrix;

[0336] A second attention loss corresponding to the original training sample is determined according to the second key-related loss, the second value-related loss and the query-related loss.

[0337] Optionally, the real key matrix, real value matrix and real query matrix each include multiple elements. For each element, if the element value corresponding to the element is a first preset value, it means that the position of the element should be paid attention to; if the element value corresponding to the element is a second preset value, it means that the position of the element should not be paid attention to.

[0338] Optionally, when the first determining module 1002 determines the second key-related loss according to the key matrix and the real key matrix, it is specifically used to:

[0339] Scaling the element value corresponding to each element in the key matrix to between a first preset value and a second preset value to obtain a scaled key matrix;

[0340] The second bond-related loss is determined based on the scaled bond matrix and the true bond matrix.

[0341] Optionally, when the first determining module 1002 determines the second key-related loss according to the scaled key matrix and the real key matrix, it is specifically used to:

[0342] For each element in the scaled key matrix, calculate the difference between the element value corresponding to the element and the element value corresponding to the target element; wherein the relative position of the target element in the real key matrix is ​​the same as the relative position of the element in the key matrix;

[0343] Calculate the sum of the differences corresponding to each element in the scaled key matrix to obtain a third sum;

[0344] Based on the third sum, a second bond-related loss is determined.

[0345] Optionally, when determining the second key-related loss according to the third sum, the first determining module 1002 is specifically configured to:

[0346] Calculate the intersection-over-union ratio of the key matrix and the real key matrix,

[0347] A product of the intersection-over-union ratio and the third sum is determined as a second bond-related loss.

[0348] Optionally, the first determining module 1002 is specifically configured to:

[0349] After the original training samples are input into the attention model, a key matrix, a value matrix and a query matrix are obtained;

[0350] determining a first attention loss based on the key matrix and the value matrix;

[0351] determining a second attention loss based on the key matrix, the value matrix, and the query matrix;

[0352] Based on the first attention loss and the second attention loss, an attention loss corresponding to the original training sample is determined.

[0353] Optionally, the acquisition module 1001 is further used for:

[0354] Obtain the sample labels corresponding to each original training sample;

[0355] For each original training sample, determine the output corresponding to the original training sample, and determine the output loss corresponding to the original training sample according to the sample label and the output corresponding to the original training sample;

[0356] Determine a first total output loss according to the output losses corresponding to the original training samples;

[0357] For each target training sample, determine the output corresponding to the target training sample, and determine the output loss corresponding to the target training sample according to the sample label and output corresponding to the target training sample;

[0358] Determine a second total output loss according to the output losses corresponding to each target training sample;

[0359] Accordingly, the fourth determining module 1005 is specifically configured to:

[0360] Determine whether to delete the second feature label according to the first total attention loss, the second total attention loss, the first total output loss, and the second total output loss.

[0361] Optionally, the fourth determining module 1005 is specifically configured to:

[0362] Determine a first total loss according to the first total attention loss and the first total output loss;

[0363] Determining a second total loss according to the second total attention loss and the second total output loss;

[0364] When the difference between the second total loss and the first total loss is smaller than a preset threshold, it is determined to delete the second feature tag.

[0365] Optionally, when the fourth determination module 1005 determines the first total loss according to the first total attention loss and the first total output loss, it is specifically configured to:

[0366] Obtaining a first weight corresponding to the first total attention loss and a second weight corresponding to the first total output loss;

[0367] A first total loss is determined according to the first total attention loss, the first total output loss, the first weight and the second weight.

[0368] Optionally, the output corresponding to each original training sample is a classification probability value, and the classification probability value is used to indicate the probability of the original training sample being classified into each category in a plurality of preset categories. When the acquisition module 1001 determines the output loss corresponding to the original training sample according to the sample label and output corresponding to the original training sample, it is specifically used to:

[0369] According to the following formula, the output loss corresponding to the original training sample is determined;

[0370]

[0371] Among them, l represents the output loss, M represents the total number of preset categories, C represents the category, P c represents the probability of being classified into the Cth class, y c represents the sample label, and γ is a constant greater than 1.

[0372] Optionally, the output corresponding to each original training sample is a classification probability value, and the classification probability value is used to indicate the probability of the original training sample being classified into each category in a plurality of preset categories. When the acquisition module 1001 determines the output loss corresponding to the original training sample according to the sample label and output corresponding to the original training sample, it is specifically used to:

[0373] According to the following formula, the output loss corresponding to the original training sample is determined;

[0374]

[0375] Among them, l represents the output loss, M represents the total number of preset categories, C represents the category, P c represents the probability of being classified into the Cth class, y c represents the sample label, and γ is a constant greater than 1.

[0376] The specific implementation principle and effect of the feature label screening device provided in the embodiment of the present application can be found in the aforementioned embodiment and will not be repeated here.

[0377] The present application embodiment also provides another feature label screening device, Fig.11 This is a schematic diagram of the structure of another feature label screening device provided in an embodiment of the present application. Fig.11 As shown, the device comprises:

[0378] An acquisition module 1101 is used to acquire a plurality of local training samples, each of which includes a plurality of local feature labels and feature values ​​corresponding to each local feature label;

[0379] An encryption module 1102 is used to encrypt, for each local training sample, the feature values ​​of each local feature label corresponding to the local training sample to obtain the original training sample, wherein the encryption method is to add Laplace noise;

[0380] The sending module 1103 is used to send multiple original training samples to the server, each original training sample includes multiple original feature labels and feature values ​​corresponding to each original feature label; so that the server obtains a key matrix and a value matrix for each original training sample after the original training sample is input into the attention model, and determines the attention loss corresponding to the original training sample according to the key matrix and the value matrix, the attention model includes an encoder and a decoder, and the key matrix and the value matrix are transmitted by the encoder to the decoder; according to the attention losses corresponding to each original training sample, a first total attention loss is determined; based on multiple target training samples and the attention model, a second total attention loss is determined; the target training sample includes at least one first feature label and feature values ​​corresponding to each first feature label; the first feature label is selected from the multiple original feature labels included in the original training sample; according to the first total attention loss and the second total attention loss, it is determined whether to delete the second feature label, wherein the second feature label is a feature label other than at least one first feature label in the multiple original feature labels.

[0381] Optionally, the encryption module 1102 is further configured to:

[0382] Normalize the feature values ​​corresponding to each local feature label to a preset dimension;

[0383] Accordingly, when the encryption module 1102 encrypts the feature values ​​of each local feature label corresponding to the local training sample to obtain the original training sample, it is specifically used to:

[0384] The normalized eigenvalues ​​corresponding to the local training samples are encrypted to obtain original training samples.

[0385] The specific implementation principle and effect of another feature label screening device provided in an embodiment of the present application can be found in the aforementioned embodiment and will not be repeated here.

[0386] Fig.12 This is a schematic diagram of the structure of the electronic device provided in this application. Fig.12 As shown, the electronic device 120 provided in this embodiment includes: at least one processor 1201 and a memory 1202. Optionally, the device 120 also includes a communication component 1203. The processor 1201, the memory 1202 and the communication component 1203 are connected via a bus 1204.

[0387] In a specific implementation process, at least one processor 1201 executes the computer-executable instructions stored in the memory 1202, so that at least one processor 1201 executes the above method.

[0388] The specific implementation process of the processor 1201 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0389] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0390] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk storage.

[0391] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0392] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0393] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0394] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0395] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0396] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0397] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0398] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0399] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0400] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0401] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A feature label screening method, characterized in that: include: Acquire multiple original training samples, each of which includes multiple original feature labels and feature values ​​corresponding to each original feature label; For each original training sample, after the original training sample is input into the attention model, a key matrix and a value matrix are obtained, and an attention loss corresponding to the original training sample is determined according to the key matrix and the value matrix, wherein the attention model includes an encoder and a decoder, and the key matrix and the value matrix are transmitted by the encoder to the decoder; Determine a first total attention loss according to the attention losses corresponding to the original training samples; Determine a second total attention loss based on multiple target training samples and an attention model; the target training samples include at least one first feature label and feature values ​​corresponding to each first feature label; The first feature label is selected from a plurality of original feature labels included in the original training sample; Determine whether to delete a second feature label according to the first total attention loss and the second total attention loss, wherein the second feature label is a feature label other than at least one first feature label among multiple original feature labels.

2. The method according to claim 1, characterized in that The attention loss includes a first attention loss; determining the attention loss corresponding to the original training sample according to the key matrix and the value matrix, including: Obtaining a true key matrix and a true value matrix corresponding to the original training sample; Determining a first key-related loss based on the key matrix and the true key matrix, and determining a first value-related loss based on the value matrix and the true value matrix; Determine a first attention loss corresponding to the original training sample according to the first key-related loss and the first value-related loss.

3. The method according to claim 2, characterized in that Determining a first key-related loss according to the key matrix and the true key matrix, and determining a first value-related loss according to the value matrix and the true value matrix, including: calculating a first mutual information between the key matrix and the real key matrix, and determining a first key-related loss according to the first mutual information; A second mutual information between the value matrix and the true value matrix is ​​calculated, and a first value correlation loss is determined according to the second mutual information.

4. The method according to claim 3, characterized in that Determining a first key-related loss according to the first mutual information includes: Calculating a first Pearson correlation coefficient between the bond matrix and the true bond matrix; Adding the first mutual information and the first Pearson correlation coefficient to obtain a first sum; determining half of the first sum as the first bond-related loss; Correspondingly, determining the first value correlation loss according to the second mutual information includes: Calculate the second Pearson correlation coefficient between the value matrix and the true value matrix; Adding the second mutual information and the second Pearson correlation coefficient to obtain a second sum; Half of the second sum is determined as the first value-related loss.

5. The method according to claim 1, characterized in that The attention loss includes a second attention loss; after the original training sample is input into the attention model, a key matrix and a value matrix are obtained, and the attention loss corresponding to the original training sample is determined according to the key matrix and the value matrix, including: After the original training samples are input into the attention model, a key matrix, a value matrix and a query matrix are obtained; Determine a second attention loss corresponding to the original training sample according to the key matrix, the value matrix and the query matrix.

6. The method according to claim 5, characterized in that Determining a second attention loss corresponding to the original training sample according to the key matrix, the value matrix and the query matrix includes: Obtaining a true key matrix, a true value matrix, and a true query matrix corresponding to the original training sample; determining a second bond-related loss based on the bond matrix and the real bond matrix; Determine a second value-related loss based on the value matrix and the true value matrix; Determining query-related losses according to the query matrix and the real query matrix; A second attention loss corresponding to the original training sample is determined according to the second key-related loss, the second value-related loss and the query-related loss.

7. The method according to claim 6, characterized in that The real key matrix, real value matrix and real query matrix each include multiple elements. For each element, if the element value corresponding to the element is a first preset value, it means that the position of the element should be paid attention to; if the element value corresponding to the element is a second preset value, it means that the position of the element should not be paid attention to.

8. The method according to claim 7, characterized in that Determining a second bond-related loss according to the bond matrix and the real bond matrix includes: Scaling the element value corresponding to each element in the key matrix to between a first preset value and a second preset value to obtain a scaled key matrix; The second bond-related loss is determined based on the scaled bond matrix and the true bond matrix.

9. The method according to claim 8, characterized in that The second bond-related loss is determined based on the scaled bond matrix and the real bond matrix, including: For each element in the scaled key matrix, calculate the difference between the element value corresponding to the element and the element value corresponding to the target element; wherein the relative position of the target element in the real key matrix is ​​the same as the relative position of the element in the key matrix; Calculate the sum of the differences corresponding to each element in the scaled key matrix to obtain a third sum; Based on the third sum, a second bond-related loss is determined.

10. The method according to claim 9, characterized in that Determining a second key-related loss based on the third sum includes: Calculate the intersection-over-union ratio of the key matrix and the real key matrix, A product of the intersection-over-union ratio and the third sum is determined as a second bond-related loss.

11. The method according to claim 1, characterized in that: After the original training sample is input into the attention model, a key matrix and a value matrix are obtained, and the attention loss corresponding to the original training sample is determined according to the key matrix and the value matrix, including: After the original training samples are input into the attention model, a key matrix, a value matrix and a query matrix are obtained; determining a first attention loss based on the key matrix and the value matrix; determining a second attention loss based on the key matrix, the value matrix, and the query matrix; Based on the first attention loss and the second attention loss, an attention loss corresponding to the original training sample is determined.

12. The method according to any one of claims 1 to 11, characterized in that: The method further comprises: Obtain the sample labels corresponding to each original training sample; For each original training sample, determine the output corresponding to the original training sample, and determine the output loss corresponding to the original training sample according to the sample label and the output corresponding to the original training sample; Determine a first total output loss according to the output losses corresponding to the original training samples; For each target training sample, determine the output corresponding to the target training sample, and determine the output loss corresponding to the target training sample according to the sample label and output corresponding to the target training sample; Determine a second total output loss according to the output losses corresponding to each target training sample; Accordingly, determining whether to delete the second feature label according to the first total attention loss and the second total attention loss includes: Determine whether to delete the second feature label according to the first total attention loss, the second total attention loss, the first total output loss, and the second total output loss.

13. The method according to claim 12, characterized in that Determining whether to delete the second feature label according to the first total attention loss, the second total attention loss, the first total output loss, and the second total output loss includes: Determine a first total loss according to the first total attention loss and the first total output loss; Determining a second total loss according to the second total attention loss and the second total output loss; When the difference between the second total loss and the first total loss is smaller than a preset threshold, it is determined to delete the second feature tag.

14. The method according to claim 13, characterized in that Determining a first total loss according to the first total attention loss and the first total output loss includes: Obtaining a first weight corresponding to the first total attention loss and a second weight corresponding to the first total output loss; A first total loss is determined according to the first total attention loss, the first total output loss, the first weight and the second weight.

15. The method according to claim 12, characterized in that The output corresponding to each original training sample is a classification probability value, and the classification probability value is used to indicate the probability of the original training sample being classified into each category in a plurality of preset categories. According to the sample label and output corresponding to the original training sample, the output loss corresponding to the original training sample is determined, including: According to the following formula, the output loss corresponding to the original training sample is determined; Among them, l represents the output loss, M represents the total number of preset categories, C represents the category, P c represents the probability of being classified into the Cth class, y c represents the sample label, and γ is a constant greater than 1.

16. The method according to claim 12, characterized in that The output corresponding to each original training sample is a classification probability value, and the classification probability value is used to indicate the probability of the original training sample being classified into each category in a plurality of preset categories. According to the sample label and output corresponding to the original training sample, the output loss corresponding to the original training sample is determined, including: According to the following formula, the output loss corresponding to the original training sample is determined; Among them, l represents the output loss, M represents the total number of preset categories, C represents the category, P c represents the probability of being classified into the Cth class, y c represents the sample label, and γ is a constant greater than 1.

17. A method for screening feature labels, characterized in that: Applied to the client, including: Acquire multiple local training samples, each local training sample including multiple local feature labels and feature values ​​corresponding to each local feature label; For each local training sample, encrypt the feature values ​​of each local feature label corresponding to the local training sample to obtain the original training sample, wherein the encryption method is to add Laplace noise; A plurality of original training samples are sent to a server, each of which includes a plurality of original feature labels and feature values ​​corresponding to each original feature label; so that the server obtains a key matrix and a value matrix for each original training sample after the original training sample is input into an attention model, and determines the attention loss corresponding to the original training sample according to the key matrix and the value matrix, wherein the attention model includes an encoder and a decoder, and the key matrix and the value matrix are transmitted from the encoder to the decoder; a first total attention loss is determined according to the attention losses corresponding to each original training sample; a second total attention loss is determined based on a plurality of target training samples and the attention model; the target training sample includes at least one first feature label and feature values ​​corresponding to each first feature label; the first feature label is selected from a plurality of original feature labels included in the original training sample; and whether to delete the second feature label is determined according to the first total attention loss and the second total attention loss, wherein the second feature label is a feature label other than at least one first feature label among the plurality of original feature labels.

18. The method according to claim 17, characterized in that The method further comprises: Normalize the feature values ​​corresponding to each local feature label to a preset dimension; Correspondingly, the feature values ​​of each local feature label corresponding to the local training sample are encrypted to obtain the original training sample, including: The normalized eigenvalues ​​corresponding to the local training samples are encrypted to obtain original training samples.

19. A feature label screening system, characterized in that: include: A server, configured to execute the method according to any one of claims 1 to 16; A client, used to execute the method described in any one of claims 17-18.

20. A feature label screening device, characterized in that: include: An acquisition module is used to acquire a plurality of original training samples, each of which includes a plurality of original feature labels and feature values ​​corresponding to the original feature labels; A first determination module is used to obtain a key matrix and a value matrix for each original training sample after the original training sample is input into an attention model, and determine the attention loss corresponding to the original training sample according to the key matrix and the value matrix, wherein the attention model includes an encoder and a decoder, and the key matrix and the value matrix are transmitted by the encoder to the decoder; A second determination module is used to determine a first total attention loss according to the attention losses corresponding to the original training samples; A third determination module is used to determine a second total attention loss based on a plurality of target training samples and an attention model; the target training samples include at least one first feature label and a feature value corresponding to each first feature label; The first feature label is selected from a plurality of original feature labels included in the original training sample; The fourth determination module is used to determine whether to delete a second feature label according to the first total attention loss and the second total attention loss, wherein the second feature label is a feature label other than at least one first feature label in multiple original feature labels.

21. A feature label screening device, characterized in that: Applied to the client, including: An acquisition module is used to acquire multiple local training samples, each of which includes multiple local feature labels and feature values ​​corresponding to each local feature label; An encryption module is used to encrypt, for each local training sample, the feature values ​​of each local feature label corresponding to the local training sample to obtain the original training sample, wherein the encryption method is to add Laplace noise; A sending module is used to send multiple original training samples to a server, each original training sample includes multiple original feature labels and feature values ​​corresponding to each original feature label; so that the server obtains a key matrix and a value matrix for each original training sample after the original training sample is input into an attention model, and determines the attention loss corresponding to the original training sample according to the key matrix and the value matrix, the attention model includes an encoder and a decoder, and the key matrix and the value matrix are transmitted by the encoder to the decoder; according to the attention losses corresponding to each original training sample, a first total attention loss is determined; based on multiple target training samples and the attention model, a second total attention loss is determined; the target training sample includes at least one first feature label and feature values ​​corresponding to each first feature label; the first feature label is screened from multiple original feature labels included in the original training sample; according to the first total attention loss and the second total attention loss, it is determined whether to delete the second feature label, wherein the second feature label is a feature label other than at least one first feature label in multiple original feature labels.

22. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 18.

23. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 18 when executed by a processor.

24. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 18 is implemented.

Images