Model training method and device, transaction detection method and device, equipment and medium

Abstract

The invention provides a model training method and device, a transaction detection method and device, equipment and a medium, and relates to the technical field of data processing and the field of finance. The training method of the model comprises the steps that a transaction sample is input into a pre-trained teacher network, sample features output by a plurality of intermediate layers of different types of the teacher network are determined, and the intermediate layers comprise network layers, except an input layer and an output layer, of the teacher network; respectively activating the sample features output by the plurality of intermediate layers to obtain a plurality of activation values; determining a target intermediate layer from the plurality of intermediate layers based on the plurality of activation values; configuring a learning intermediate layer matched with the target intermediate layer for the initial student network; and training the initial student network by using a target loss function and the training sample to obtain a trained student network, the target loss function being used for evaluating the difference between the features output by the target intermediate layer and the features output by the learning intermediate layer.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of data processing and the field of finance, and more particularly to a model training method, a transaction detection method, an apparatus, a device, and a medium. BACKGROUND

[0002] With the rapid development of the financial industry, especially the popularity of electronic payment and cross-border payment, financial platforms are facing increasingly complex transaction behaviors, and it is particularly important to identify abnormal transactions from complex transactions. Since many transactions currently occur on mobile terminal devices, large-scale deep learning models need to be deployed in resource-limited devices or network environments, and the computing resource requirements of large-scale deep learning models are large, which cannot be efficiently run in some low-resource environments.

[0003] In related technologies, the knowledge of a large-scale deep learning model can be transferred to a lightweight student network through knowledge distillation to reduce model computing resource consumption and improve real-time performance. However, the existing standard distillation model mainly focuses on matching the final output layer, and the learning efficiency and prediction accuracy of the student network need to be improved. SUMMARY

[0004] In view of the above problems, the present disclosure provides a model training method, a transaction detection method, an apparatus, a device, and a medium.

[0005] According to a first aspect of the present disclosure, a model training method is provided, comprising: inputting a transaction sample into a pre-trained teacher network, determining sample features output by each of a plurality of intermediate layers of different types of the teacher network, the intermediate layers including network layers of the teacher network other than the input layer and the output layer; respectively activating the sample features output by each of the plurality of intermediate layers to obtain a plurality of activation values; determining a target intermediate layer from the plurality of intermediate layers based on the plurality of activation values; configuring an initial student network with a learning intermediate layer matching the target intermediate layer; training the initial student network using a target loss function and a training sample to obtain a trained student network, the target loss function being used to evaluate the difference between the features output by the target intermediate layer and the features output by the learning intermediate layer.

[0006] According to an embodiment of the present disclosure, determining the target intermediate layer from the plurality of intermediate layers based on the plurality of activation values comprises: determining a plurality of candidate intermediate layers from the plurality of intermediate layers based on the plurality of activation values; screening the plurality of candidate intermediate layers using transaction verification pairs to determine the target intermediate layer from the plurality of candidate intermediate layers, wherein the data similarity between two verification samples of the same transaction type included in the transaction verification pair is lower than a predetermined threshold.

[0007] According to an embodiment of the present disclosure, the target intermediate layer is determined from the plurality of candidate intermediate layers by transaction verification, comprising: inputting two verification samples into the teacher network respectively, and obtaining the activation values of the two verification samples at the candidate intermediate layer for any candidate intermediate layer; and determining the candidate intermediate layer as the target intermediate layer when the difference between the activation values of the two verification samples at the candidate intermediate layer meets a predetermined difference condition.

[0008] According to an embodiment of the present disclosure, the method further comprises: performing structure type identification on the plurality of to-be-screened intermediate layers of different types of the teacher network to obtain an identification result; and determining an intermediate layer matching a predetermined structure type from the plurality of to-be-screened intermediate layers based on the identification result; wherein the predetermined structure type comprises at least one of the following: a structure type for performing time sequence feature extraction, a structure type for performing key feature extraction, and a structure type for capturing context-related features.

[0009] According to an embodiment of the present disclosure, the student network further comprises an adaptation layer configured before the learning intermediate layer, and the adaptation layer is configured to perform dimensionality increasing processing or dimensionality reducing processing on the learning intermediate layer when the dimension of the target intermediate layer does not match the dimension of the learning intermediate layer.

[0010] According to an embodiment of the present disclosure, the target loss function further comprises a hard label loss and a soft label loss, the hard label loss representing the difference between the output of the student network and the real label, and the soft label loss representing the difference between the output of the teacher network and the output of the student network; and the initial student network is trained by using the target loss function to obtain a trained student network, comprising: determining a first weight of the hard label loss, a second weight of the soft label loss, and a third weight of the intermediate layer loss according to the training stage of the initial student network, the training stage representing different time stages of the initial student network in the complete training process; and training the initial student network by using the hard label loss with the first weight, the soft label loss with the second weight, and the intermediate layer loss with the third weight to obtain the trained student network; wherein the weight proportions of the first weight, the second weight, and the third weight are different at different time stages, and the sum of the first weight, the second weight, and the third weight is 1.

[0011] The second aspect of the present disclosure provides a transaction detection method, comprising: obtaining transaction data of a target transaction; inputting the transaction data of the target transaction into a student network to output a detection result of the target transaction, wherein the student network is trained by the training method of the model.

[0012] The third aspect of the present disclosure provides a training device of a model, a first determination module configured to input a transaction sample into a pre-trained teacher network, determine sample features output by each of a plurality of different types of intermediate layers of the teacher network, the intermediate layers including network layers of the teacher network other than input layers and output layers; an activation module configured to activate the sample features output by each of the plurality of intermediate layers respectively to obtain a plurality of activation values; a second determination module configured to determine a target intermediate layer from the plurality of intermediate layers based on the plurality of activation values; a configuration module configured to configure an initial student network with learning intermediate layers matched with the target intermediate layer; and a training module configured to train the initial student network using a target loss function and a training sample to obtain a trained student network, the target loss function configured to evaluate a difference between features output by the target intermediate layer and features output by the learning intermediate layers.

[0013] The fourth aspect of the present disclosure provides a device for business detection, the device comprising: an acquisition module configured to acquire transaction data of a target transaction; and a detection module configured to input the transaction data of the target transaction into a student network to output a detection result of the target transaction, wherein the student network is trained according to the model training method of the preceding claim.

[0014] The fifth aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory configured to store one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method.

[0015] The sixth aspect of the present disclosure further provides a computer-readable storage medium having stored thereon a computer program or instructions, wherein the computer program or instructions, when executed by a processor, implement the steps of the method.

[0016] The seventh aspect of the present disclosure further provides a computer program product comprising a computer program or instructions, wherein the computer program or instructions, when executed by a processor, implement the steps of the method.

[0017] According to the embodiments of the present disclosure, by inputting a transaction sample into a teacher network and analyzing activation values using sample features of intermediate layers, a target intermediate layer sensitive to abnormal transactions is determined, and a learning intermediate layer matched with the target intermediate layer is set in an initial student network, and an intermediate layer loss is used to prompt the student to not only focus on the final prediction result during the training process, but also pay attention to learning the feature representation of the intermediate layers of the teacher network. The student network learning can effectively learn the key knowledge of the teacher network, thereby realizing effective knowledge transfer, and improving the learning efficiency and prediction accuracy of the initial student network itself. BRIEF DESCRIPTION OF DRAWINGS

[0018] The foregoing contents, as well as other objects, features, and advantages of this disclosure, will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0019] Figure 1 The illustration shows an application scenario of the model training method and the transaction detection method according to embodiments of the present disclosure;

[0020] Figure 2 A flowchart illustrating a training method for a model according to an embodiment of the present disclosure is shown schematically.

[0021] Figure 3 This illustration schematically shows a process diagram for determining a target intermediate layer according to an embodiment of the present disclosure;

[0022] Figure 4 A flowchart illustrating a transaction detection method according to an embodiment of the present disclosure is shown schematically.

[0023] Figure 5 This schematic diagram illustrates a structural block diagram of a training apparatus for a model according to an embodiment of the present disclosure;

[0024] Figure 6 This schematic diagram illustrates a structural block diagram of a transaction detection apparatus according to an embodiment of the present disclosure;

[0025] Figure 7 A block diagram of an electronic device suitable for implementing a model training method according to an embodiment of the present disclosure is shown schematically. Detailed Implementation

[0026] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0027] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0028] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0029] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0030] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0031] In the technical solution disclosed herein, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, and displayed data) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with relevant laws, regulations, and standards, and necessary measures have been taken to ensure that they do not violate public order and good morals. Corresponding operation entry points are provided for users to choose to authorize or refuse.

[0032] Figure 1 The illustration shows an application scenario of the model training method and the transaction detection method according to embodiments of the present disclosure.

[0033] like Figure 1 As shown, application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0034] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0035] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0036] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0037] It should be noted that the model training method and transaction detection provided in this embodiment can generally be executed by server 105. Correspondingly, the model training device and transaction detection device provided in this embodiment can generally be located in server 105. The model training method and transaction detection method provided in this embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the model training device and transaction detection device provided in this embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105.

[0038] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0039] Figure 2 A flowchart illustrating a training method for a model according to an embodiment of the present disclosure is shown schematically.

[0040] The training method for this model includes operations S210 to S250.

[0041] In operation S210, transaction samples are input into a pre-trained teacher network, and the sample features output by multiple intermediate layers of different types of the teacher network are determined. The intermediate layers include network layers of the teacher network other than the input and output layers.

[0042] In operation S220, the sample features output by each of the multiple intermediate layers are activated to obtain multiple activation values.

[0043] In operation S230, the target intermediate layer is determined from multiple intermediate layers based on multiple activation values.

[0044] In operation S240, the initial student network configuration is matched with the target intermediate layer for learning intermediate layer.

[0045] In operation S250, the initial student network is trained using the target loss function and training samples to obtain the trained student network. The target loss function is used to evaluate the difference between the features output by the target intermediate layer and the features output by the learned intermediate layer.

[0046] Transaction samples can be generated using publicly available datasets or historical transaction datasets. Transaction samples may include information such as transaction amount, transaction time, transaction location, information of both parties, and transaction type. Transaction samples can be categorized into normal transaction samples and abnormal transaction samples. Abnormal transaction samples refer to transactions that do not conform to normal transaction patterns and may involve fraud, violations, or other abnormal behavior. Normal transaction samples represent transactions that conform to normal business logic and transaction habits.

[0047] The teacher network can be trained using a training set from transaction samples. The teacher network can be a large, relatively complex neural network, such as a deep convolutional neural network (CNN), a recurrent neural network (RNN), or a Transformer. This disclosure does not limit the type of teacher network. In transaction detection tasks, the teacher network may have learned various complex features from the transaction data, enabling it to distinguish between normal and abnormal transactions effectively.

[0048] Activation values ​​represent the output values ​​of neurons or channels after processing the sample features of the intermediate layer through an activation function, given input data.

[0049] Intermediate layers refer to hidden layers other than the input and output layers. Different intermediate layers may learn features at different levels of abstraction. The activity level of an intermediate layer can be determined based on its activation value, and an intermediate layer with higher activity can be selected as the target intermediate layer.

[0050] The initial student network can be a shallow neural network. A learning intermediate layer is designed within the initial student network to perform feature matching with the target intermediate layer of the teacher network. During training, the learning intermediate layer adjusts its parameters using intermediate layer loss to make the features output by the initial student network as similar as possible to the features of the target intermediate layer of the teacher network.

[0051] Intermediate layer loss is used to measure the difference between the features of the teacher's network objective intermediate layer and the features of the student's network learning intermediate layer. Intermediate layer loss can be calculated using metrics such as mean squared error and cosine similarity.

[0052] According to embodiments of this disclosure, by inputting transaction samples into the teacher network and performing activation value analysis using the sample features of the intermediate layers, a target intermediate layer sensitive to abnormal transactions is identified. A learning intermediate layer matching the target intermediate layer is then set in the initial student network. The intermediate layer loss is used to encourage students to focus not only on the final prediction result during training but also on learning the feature representations of the teacher network's intermediate layers. This enables the student network to effectively learn the key knowledge of the teacher network, thereby achieving effective knowledge transfer and improving the learning efficiency and prediction accuracy of the initial student network itself.

[0053] According to embodiments of this disclosure, determining a target intermediate layer from multiple intermediate layers based on multiple activation values ​​may include: determining multiple candidate intermediate layers from multiple intermediate layers based on multiple activation values; filtering the multiple candidate intermediate layers using a transaction verification pair to determine the target intermediate layer from the multiple candidate intermediate layers, wherein the data similarity between two verification samples of the same transaction type included in the transaction verification pair is lower than a predetermined threshold.

[0054] For example, the average information entropy of each intermediate layer activation value can be calculated. Information entropy is a measure of data uncertainty. The higher the information entropy of the activation value, the richer the features extracted by that layer and the greater the discriminative power. The intermediate layers are sorted in descending order of information entropy, and the intermediate layers with higher rankings are selected as subsequent intermediate layers.

[0055] Transaction types can include, for example, transfers, purchases, and investments. Transaction verification pairs can be, for example, sample pairs consisting of normal and abnormal transactions of the same type. The predetermined threshold can be determined, for example, by the mean difference between normal and abnormal transaction samples.

[0056] According to embodiments of this disclosure, using transaction verification to screen multiple candidate intermediate layers and determining a target intermediate layer from multiple candidate intermediate layers may include: inputting two verification samples into the teacher network respectively; obtaining the activation values ​​of the two verification samples in the candidate intermediate layer for any candidate intermediate layer; and determining the candidate intermediate layer as the target intermediate layer if the difference between the activation values ​​of the two verification samples in the candidate intermediate layer satisfies a predetermined difference condition.

[0057] The difference between the activation value distributions of two validation samples in the candidate intermediate layer can be calculated. For example, the difference between the two sets of activation values ​​can be measured by the KL divergence (Kullback-Leibler divergence) of the activation values ​​of the two validation samples in the candidate intermediate layer. In this case, the predetermined difference condition can be a predetermined divergence threshold. When the KL divergence is greater than the predetermined divergence threshold, the candidate intermediate layer is determined as the target intermediate layer.

[0058] This disclosure is not limited to this. The difference between two groups can also be determined by calculating the mean difference, variance difference, etc., of the activation values. When the difference between the two groups exceeds a predetermined difference threshold, the candidate intermediate layer is determined as the target intermediate layer. Alternatively, all candidate intermediate layers can be sorted according to the difference index, and the intermediate layer with the largest target number of differences can be selected as the candidate intermediate layer.

[0059] According to embodiments of this disclosure, candidate intermediate layers are determined from multiple intermediate layers based on multiple activation values. Since the activation values ​​of different intermediate layers contain feature information of different levels and types, potential sensitive layers can be quickly located. Then, based on the difference between the activation value distributions of two verification samples in the candidate intermediate layers, the target intermediate layer is selected from the candidate intermediate layers, which can effectively enhance the rigor and reliability of the determination of the target intermediate layer.

[0060] Figure 3 A schematic diagram illustrating the process of determining a target intermediate layer according to an embodiment of the present disclosure is shown.

[0061] like Figure 3 As shown, transaction verification pairs 310 are input into a teacher network 320, which includes an input layer, multiple candidate intermediate layers, and an output layer. For any candidate intermediate layer, the activation values ​​330 of the two verification samples in the candidate intermediate layer are obtained. If the difference between the activation values ​​of the two verification samples in the candidate intermediate layer satisfies a predetermined difference condition 340, the candidate intermediate layer is determined as the target intermediate layer 350.

[0062] According to embodiments of this disclosure, before performing operation S210, the process may further include: identifying the structural types of multiple intermediate layers of different types to be screened in the teacher network to obtain identification results; and based on the identification results, determining the intermediate layer among the multiple intermediate layers to be screened that matches a predetermined structural type. The predetermined structural type includes at least one of the following: a structural type for performing temporal feature extraction, a structural type for performing key feature extraction, and a structural type for capturing contextual features.

[0063] For example, if the intermediate layer contains recurrent units, it can be determined that the layer may belong to the structure type used for temporal feature extraction, such as a long short-term memory network layer, a recurrent neural network layer, etc.

[0064] If the intermediate layer contains convolutional operations, and the parameters such as the size of the convolutional kernel and the stride are designed to filter local key features, then it can be preliminarily determined that the layer belongs to the structure type used for key feature extraction, such as a convolutional layer.

[0065] If the intermediate layer uses an attention mechanism to capture contextual relationships by calculating attention weights between different positions, then the layer can be identified as a structure type used to capture contextual relationship features, such as an attention layer.

[0066] According to embodiments of this disclosure, by identifying the structural types of multiple intermediate layers of different types in a teacher network and determining the intermediate layer that matches the predetermined structural type based on the identification results, the number of analysis layers can be reduced while ensuring that the selected intermediate layer can efficiently perform its specific function, avoiding insufficient or inaccurate feature extraction due to functional mismatch of intermediate layers.

[0067] For example, for any candidate intermediate layer, the parameters of the teacher network can be frozen, and normal transaction samples can be used as input data for the teacher network. The input data is then processed iteratively multiple times until the activation value of the candidate intermediate layer is maximized, thus obtaining a synthetic abnormal transaction sample. Based on the synthetic abnormal transaction samples corresponding to each of the multiple candidate intermediate layers, the target intermediate layer is determined.

[0068] For example, for any candidate intermediate layer, the channel or neuron in that candidate intermediate layer whose activation value is significantly higher on abnormal samples can be selected as the target. Real normal transaction samples are chosen as the starting point for optimization, and in each iteration, the gradient ascent algorithm can be used to adjust the input data, causing the activation value of the candidate intermediate layer to gradually increase.

[0069] By continuously iterating over the input data until the activation values ​​no longer increase significantly or the generated samples tend to stabilize, the resulting input data becomes the synthetic anomalous transaction sample corresponding to that candidate intermediate layer.

[0070] For example, the synthetic abnormal transaction samples generated by each candidate intermediate layer can be further judged to determine whether the abnormal transaction sample conforms to a reasonable abnormal transaction pattern. If so, the candidate intermediate layer is considered as the target intermediate layer.

[0071] By initially screening the differences in activation values ​​between abnormal and normal transaction samples, potential sensitive layers can be quickly identified. Subsequently, activation value maximization analysis allows for functional verification of candidate intermediate layers. If business-interpretable synthetic abnormal transaction samples can be iteratively generated starting from normal transaction samples, it proves that the intermediate layer indeed encodes meaningful anomaly discrimination features, rather than irrelevant noise. This provides a clear and reliable knowledge transfer anchor for subsequent knowledge distillation, ensuring that the student network can efficiently inherit the teacher's core abnormal transaction detection capabilities.

[0072] For example, for any iteration, using normal transaction samples as input data to the teacher network and performing multiple iterations on the input data may include: inputting the current input data into the teacher network to obtain the activation value of the candidate intermediate layer in the current iteration; determining the target gradient for updating the current input data based on the current activation value and the previous activation value; and updating the current input data using the target gradient to obtain the input data for the next iteration.

[0073] The current input data can be fed into the teacher network with frozen parameters, and a complete forward computation can be run to obtain the activation values ​​generated by the candidate intermediate layer under the current input.

[0074] For example, a loss function can be defined, and the gradient of the loss with respect to the input data can be calculated based on the loss function to obtain the target gradient.

[0075] For example, the loss function can be seen in formula (1):

[0076] (1)

[0077] in, Indicates loss, This represents the input data, where A represents the activation value. Indicates the first One candidate intermediate layer, Indicates the first The first candidate intermediate layer There are 1 target channel, where N represents the total number of all activated values ​​in the target channel.

[0078] (2)

[0079] in, This represents the target gradient.

[0080] The target gradient can be used to update the input data to obtain the next input data.

[0081] For example, constraints can also be imposed on the input data after each update to ensure that each of its feature values ​​is within a reasonable range; for example, the transaction amount cannot be negative.

[0082] By updating the input data through the target gradient, the input data gradually evolves into a synthetic anomalous sample that can greatly stimulate the response of a specific candidate intermediate layer in the teacher network, while maintaining the rationality of the data. This allows for the reverse deduction of the most sensitive feature pattern encoded by the intermediate layer, providing an intuitive and interpretable basis for locating the key feature extraction layer.

[0083] For example, determining the target intermediate layer based on the synthetic abnormal transaction samples of multiple candidate intermediate layers may include: determining the update features of the normal transaction samples based on the synthetic abnormal transaction samples and normal transaction samples; and determining the candidate intermediate layer corresponding to the synthetic abnormal transaction sample as the target intermediate layer if the update features meet a preset update condition. Alternatively, matching the synthetic abnormal transaction samples with abnormal transaction samples, and determining the candidate intermediate layer corresponding to the synthetic abnormal transaction sample as the target intermediate layer if the matching degree between the abnormal transaction samples and the synthetic abnormal transaction samples meets a preset matching condition.

[0084] Based on the feature differences between the synthesized abnormal transaction samples and normal transaction samples, the updated features, as well as the direction and magnitude of the updates, can be determined in the normal transaction samples.

[0085] By identifying one or more features with large update magnitudes as key update features, it is determined whether the key update features and their change direction meet the preset update conditions.

[0086] Preset update conditions could include, for example, a significant increase in the transaction amount characteristic value, an abnormally high increase in the transaction frequency characteristic value, or whether the changed characteristic value is within a reasonable but abnormal range.

[0087] If the key updated features of the synthesized abnormal transaction sample meet the preset conditions, it indicates that the candidate intermediate layer has successfully captured business-understandable fraud features, and therefore can be identified as the target intermediate layer. Conversely, if the main updated features are disorganized or do not conform to any fraud logic, for example, if the main updated features are irrelevant such as "user age", then the candidate intermediate layer is excluded.

[0088] For example, the similarity between the synthetic abnormal transaction samples generated by the candidate intermediate layer and the real abnormal samples in the abnormal sample library can be calculated. For example, matching can be performed using cosine similarity, Euclidean distance, or based on preset rules.

[0089] Preset matching criteria could be, for example, a similarity threshold. If the matching degree between the synthesized abnormal transaction sample and the real abnormal sample library meets the preset criteria, it proves that the candidate intermediate layer has captured real and universal abnormal features, and therefore can be identified as the target intermediate layer. Conversely, if the synthesized abnormal transaction sample is dissimilar to any real fraud, the candidate intermediate layer has low practical value and should be excluded.

[0090] By synthesizing updated features of abnormal transaction samples and determining whether they meet preset conditions, a target intermediate layer that effectively reflects abnormal characteristics is accurately selected. The synthesized abnormal samples are matched with real abnormal samples, and the target intermediate layer is determined based on the matching degree, ensuring that the selected intermediate layer highly matches the real abnormal scenario, thereby improving the accuracy and reliability of abnormal transaction detection.

[0091] According to embodiments of this disclosure, the student network further includes an adaptation layer configured before the learning intermediate layer. The adaptation layer is used to perform dimensionality-upgrading or dimensionality-downgrading on the learning intermediate layer when the dimensions of the target intermediate layer do not match the dimensions of the learning intermediate layer.

[0092] For example, if the dimension of the target intermediate layer of the student network is greater than the dimension of the learning intermediate layer, it can be increased in dimension by adding 1x1 convolutional or fully connected layers.

[0093] If the dimension of the target intermediate layer of the student network is smaller than the dimension of the learning intermediate layer, dimensionality reduction can also be achieved by adding 1x1 convolutional or fully connected layers.

[0094] According to embodiments of this disclosure, the compatibility problem caused by differences in intermediate layer dimensions between different network structures can be solved by the adaptation layer, ensuring the effective transmission and fusion of information in different dimensional feature spaces.

[0095] According to embodiments of this disclosure, the target loss function further includes hard-label loss and soft-label loss, where hard labels characterize the difference between the output of the student network and the true labels, and soft-label loss characterizes the difference between the output of the teacher network and the output of the student network.

[0096] Training the initial student network using the target loss function to obtain the trained student network can include: determining the first weight of the hard label loss, the second weight of the soft label loss, and the third weight of the intermediate layer loss based on the training stage of the initial student network, where the training stage represents different time stages of the initial student network in the complete training process; training the initial student network using the first weight of the hard label loss, the second weight of the soft label loss, and the third weight of the intermediate layer loss to obtain the trained student network; wherein the weight ratio of the first weight, the second weight, and the third weight is different in different time stages, and the sum of the first weight, the second weight, and the third weight is 1.

[0097] For example, the target loss function L total See formula (3):

[0098] L total = αL soft + β * L feat + γ * L hard(3)

[0099] Among them, L soft L represents the loss for soft targets. feat L represents the intermediate layer loss. hard α represents the hard-label loss, and β and γ represent adjustable weight coefficients.

[0100] During training, α, β, and γ can be dynamically adjusted based on evaluation metrics, such as the F1 score on the validation set. For example, initially, the focus is on learning teacher knowledge patterns (with a larger β), while later, the focus shifts to fitting the true labels (with a larger γ).

[0101] For example, in the early stages of training, the focus is on knowledge transfer for teachers. For instance, if the current training rounds are less than 30% of the preset maximum total training rounds, β = 0.8 and γ = 0.1 can be used.

[0102] In the middle of training, for example, when the current training epoch is greater than 30% of the preset total training epoch limit but less than 70% of the preset total training epoch limit, a balance is struck between transfer learning and true label learning.

[0103] β = 0.4 - (current_epoch / max_epochs)*0.2

[0104] γ = 0.6 + (current_epoch / max_epochs)*0.2

[0105] Where current_epoch represents the current training epoch, and max_epochs represents the preset upper limit of the total number of training epochs.

[0106] In the later stages of training, for example, when the current training rounds exceed 70% of the preset maximum total training rounds, β = 0.1 and γ = 0.8 can be used.

[0107] For example, during training overload, if the F1 score does not improve for several consecutive rounds, then γ is increased, γ = min(γ * 1.1, 0.9).

[0108] α is usually fixed and can take values ​​from 0.1 to 0.3. The minimum value of β is set to 0.05 and the minimum value of γ is set to 0.1 to prevent the complete neglect of a certain part of the loss.

[0109] According to embodiments of this disclosure, an adaptive weight adjustment strategy can enable student networks to learn not only the teacher's final decision, but also key intermediate representations for identifying fraud, accelerating convergence and potentially improving generalization, especially in capturing key minority class patterns when data is imbalanced.

[0110] Figure 4A flowchart illustrating a transaction detection method according to an embodiment of this disclosure is shown schematically.

[0111] The transaction detection method includes: Operation S410 to Operation S420.

[0112] In operation S410, the transaction data of the target transaction is obtained.

[0113] When operating the S420, the transaction data of the target transaction is input into the student network, and the detection result of the target transaction is output.

[0114] According to embodiments of this disclosure, the student network is trained using a model training method that performs operations S210 to S230.

[0115] For example, the student network can be deployed in an edge device. Whenever a transaction occurs, the transaction data of the target transaction is input into the student network for real-time inference. When predicting the target transaction, the student network can quickly determine whether the target transaction is an abnormal transaction.

[0116] According to embodiments of this disclosure, since the student network trained by knowledge distillation is lightweight, it can run on edge devices (such as POS terminals, mobile devices), reducing data transmission and processing latency, saving computing resources in cloud computing and data centers, and reducing costs.

[0117] Figure 5 A schematic block diagram of a training apparatus for a model according to an embodiment of the present disclosure is shown.

[0118] like Figure 5 As shown, the training device 500 for the model includes a first determining module 510, an activation module 520, a second determining module 530, a configuration module 540, and a training module 550.

[0119] The first determining module 510 is used to input transaction samples into a pre-trained teacher network and determine the sample features output by each of the multiple intermediate layers of different types of the teacher network. The intermediate layers include network layers of the teacher network other than the input layer and the output layer.

[0120] The activation module 520 is used to activate the sample features output by each of the multiple intermediate layers to obtain multiple activation values.

[0121] The second determining module 530 is used to determine the target intermediate layer from multiple intermediate layers based on multiple activation values.

[0122] Configuration module 540 is used to configure the initial student network with a learning middleware layer that matches the target middleware layer.

[0123] Training module 550 is used to train the initial student network using the target loss function and training samples to obtain the trained student network. The target loss function is used to evaluate the difference between the features output by the target intermediate layer and the features output by the learned intermediate layer.

[0124] According to embodiments of this disclosure, the second determining module 530 includes a first determining submodule and a second determining submodule.

[0125] The first determination submodule is used to determine multiple candidate intermediate layers from multiple intermediate layers based on multiple activation values.

[0126] The second determination submodule is used to filter multiple candidate intermediate layers using transaction verification pairs and determine the target intermediate layer from the multiple candidate intermediate layers, wherein the data similarity between two verification samples of the same transaction type included in the transaction verification pair is lower than a predetermined threshold.

[0127] According to embodiments of this disclosure, the second determining submodule includes a first determining unit and a second determining unit.

[0128] The first determining unit is used to input the two verification samples into the teacher network respectively, and for any candidate intermediate layer, obtain the activation values ​​of the two verification samples in the candidate intermediate layer respectively.

[0129] The second determining unit is used to determine the candidate intermediate layer as the target intermediate layer when the difference between the activation values ​​of the two verification samples in the candidate intermediate layer meets the predetermined difference condition.

[0130] According to embodiments of this disclosure, the training device 500 for the model includes an identification module and a third determination module.

[0131] The identification module is used to identify the structural type of multiple intermediate layers of different types in the teacher network and obtain the identification results.

[0132] The third determining module is used to determine, based on the recognition results, an intermediate layer that matches a predetermined structure type from among multiple intermediate layers to be screened; wherein the predetermined structure type includes at least one of the following: a structure type for performing temporal feature extraction, a structure type for performing key feature extraction, and a structure type for capturing contextual features.

[0133] According to embodiments of this disclosure, the student network further includes an adaptation layer configured before the learning intermediate layer. The adaptation layer is used to perform dimensionality-upgrading or dimensionality-downgrading on the learning intermediate layer when the dimensions of the target intermediate layer do not match the dimensions of the learning intermediate layer.

[0134] According to embodiments of this disclosure, the target loss function further includes hard-label loss and soft-label loss, where hard labels characterize the difference between the output of the student network and the true labels, and soft-label loss characterizes the difference between the output of the teacher network and the output of the student network.

[0135] Training module 550 includes a third determination module and a training submodule.

[0136] The third determination module is used to determine the first weight of the hard label loss, the second weight of the soft label loss, and the third weight of the intermediate layer loss based on the training stage of the initial student network. The training stage represents different time stages of the initial student network in the complete training process.

[0137] The training submodule is used to train the initial student network using the hard label loss with the first weight, the soft label loss with the second weight, and the intermediate layer loss with the third weight, to obtain the trained student network. The weight ratios of the first weight, the second weight, and the third weight are different at different time stages, and the sum of the first weight, the second weight, and the third weight is 1.

[0138] It should be noted that the model training device part in the embodiments of this disclosure corresponds to the model training method part of this disclosure. The specific description of the model training device is referred to in the model training method part, and will not be repeated here.

[0139] Figure 6 A schematic block diagram of a transaction detection apparatus according to an embodiment of the present disclosure is shown.

[0140] like Figure 6 As shown, the service detection device 600 includes an acquisition module 610 and a detection module 620.

[0141] The acquisition module 610 is used to acquire the transaction data of the target transaction.

[0142] The detection module 620 is used to input the transaction data of the target transaction into the student network and output the detection result of the target transaction, wherein the student network is trained according to the model training method described above.

[0143] Any one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure, or at least part of the functions of any one or more of them, can be implemented in one module. Any one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure can be implemented by dividing them into multiple modules. Any one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure can be at least partially implemented as hardware circuitry, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a System-on-Chip, a System-on-a-Substrate, a System-on-Package, an Application-Specific Integrated Circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure can be at least partially implemented as computer program modules, which, when run, can perform corresponding functions.

[0144] For example, any plurality of modules among the first determining module 510, activation module 520, second determining module 530, configuration module 540, and training module 550 may be combined into one module, or any one of these modules may be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of other modules and implemented in one module. At least one of the first determining module 510, activation module 520, second determining module 530, configuration module 540, and training module 550 according to embodiments of the present disclosure may be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging the circuitry, or implemented in any one of software, hardware, and firmware methods, or in a suitable combination of any of these methods. Alternatively, at least one of the first determining module 510, activation module 520, second determining module 530, configuration module 540 and training module 550 may be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0145] Figure 7 A block diagram of an electronic device suitable for implementing a model training method according to an embodiment of the present disclosure is shown schematically.

[0146] like Figure 7As shown, an electronic device 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage portion 708 into a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0147] RAM 703 stores various programs and data required for the operation of electronic device 700. Processor 701, ROM 702, and RAM 703 are interconnected via bus 704. Processor 701 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 702 and / or RAM 703. It should be noted that programs may also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in one or more memories.

[0148] According to embodiments of this disclosure, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to a bus 704. The electronic device 700 may also include one or more of the following components connected to the input / output (I / O) interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output (I / O) interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0149] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0150] According to embodiments of this disclosure, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 702 and / or RAM 703 and / or one or more memories other than ROM 702 and RAM 703 described above.

[0151] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to enable the computer system to implement the training method for the model provided in the embodiments of this disclosure.

[0152] When the computer program is executed by the processor 701, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0153] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 709, and / or installed from a removable medium 711. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0154] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0155] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on a user's computing device, partially on a user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0156] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0157] Those skilled in the art will understand that the features described in the various embodiments of this disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments of this disclosure can be combined and / or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0158] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A method for training a model, characterized in that, The method includes: The transaction samples are input into a pre-trained teacher network, and the sample features output by each of the multiple intermediate layers of different types of the teacher network are determined. The intermediate layers include the network layers of the teacher network other than the input layer and the output layer. The sample features output by each of the multiple intermediate layers are activated separately to obtain multiple activation values; Based on the multiple activation values, a target intermediate layer is determined from the multiple intermediate layers; A learning intermediate layer that matches the initial student network configuration with the target intermediate layer; The initial student network is trained using a target loss function and training samples to obtain a trained student network. The target loss function is used to evaluate the difference between the features output by the target intermediate layer and the features output by the learned intermediate layer.

2. The training method according to claim 1, characterized in that, The step of determining the target intermediate layer from the multiple intermediate layers based on the multiple activation values ​​includes: Based on the multiple activation values, multiple candidate intermediate layers are determined from the multiple intermediate layers; The transaction verification pair is used to screen multiple candidate intermediate layers, and the target intermediate layer is determined from the multiple candidate intermediate layers, wherein the data similarity between two verification samples of the same transaction type included in the transaction verification pair is lower than a predetermined threshold.

3. The method according to claim 2, characterized in that, Using transaction verification to filter multiple candidate intermediate layers, and determining the target intermediate layer from the multiple candidate intermediate layers, includes: The two verification samples are input into the teacher network respectively, and for any candidate intermediate layer, the activation values ​​of the two verification samples in the candidate intermediate layer are obtained respectively. If the difference between the activation values ​​of the two verification samples in the candidate intermediate layer satisfies a predetermined difference condition, the candidate intermediate layer is determined as the target intermediate layer.

4. The method according to claim 1, characterized in that, The method further includes: The structural type of multiple intermediate layers of different types to be screened in the teacher network is identified to obtain the identification results; as well as Based on the identification results, the intermediate layer that matches the predetermined structure type is determined from the multiple intermediate layers to be screened; The predetermined structure type includes at least one of the following: a structure type for extracting temporal features, a structure type for extracting key features, and a structure type for capturing contextual features.

5. The method according to claim 1, characterized in that, The student network also includes an adaptation layer, which is configured before the learning intermediate layer. The adaptation layer is used to perform dimensionality-upgrading or dimensionality-downgrading on the learning intermediate layer when the dimensions of the target intermediate layer do not match the dimensions of the learning intermediate layer.

6. The method according to claim 1, characterized in that, The target loss function also includes hard label loss and soft label loss. The hard label represents the difference between the output of the student network and the true label, and the soft label loss represents the difference between the output of the teacher network and the output of the student network. The process of training the initial student network using the target loss function to obtain the trained student network includes: Based on the training stage of the initial student network, the first weight of the hard label loss, the second weight of the soft label loss, and the third weight of the intermediate layer loss are determined. The training stage represents different time stages of the initial student network in the complete training process. The initial student network is trained using the hard label loss with the first weight, the soft label loss with the second weight, and the intermediate layer loss with the third weight to obtain the trained student network. The weight ratios of the first weight, the second weight, and the third weight are different at different time stages, and the sum of the first weight, the second weight, and the third weight is 1.

7. A transaction detection method, characterized in that, The method includes: Obtain the transaction data of the target transaction; The transaction data of the target transaction is input into the student network, and the detection result of the target transaction is output. The student network is trained by the model training method according to any one of claims 1 to 6.

8. A training device for a model, characterized in that, The device includes: The first determining module is used to input transaction samples into a pre-trained teacher network and determine the sample features output by each of the multiple intermediate layers of different types of the teacher network, wherein the intermediate layers include network layers of the teacher network other than the input layer and the output layer. The activation module is used to activate the sample features output by each of the multiple intermediate layers to obtain multiple activation values. The second determining module is used to determine a target intermediate layer from the multiple intermediate layers based on the multiple activation values; The configuration module is used to configure the initial student network with a learning intermediate layer that matches the target intermediate layer; The training module is used to train the initial student network using a target loss function and training samples to obtain a trained student network. The target loss function is used to evaluate the difference between the features output by the target intermediate layer and the features output by the learning intermediate layer.

9. A device for business detection, characterized in that, The device includes: The acquisition module is used to acquire transaction data for the target transaction. The detection module is used to input the transaction data of the target transaction into the student network and output the detection result of the target transaction, wherein the student network is trained by the model training method according to any one of claims 1 to 6.

10. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

11. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 7.

12. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 7.

Images