A method for identifying financial fraud based on multimodal attention networks
By utilizing the self-attention and guided attention modules of a multimodal attention network, the problem of capturing multimodal data interaction information in existing technologies is solved, achieving higher accuracy in identifying financial fraud and dynamically adjusting modal influence to improve the identification effect.
Patent Information
- Application Number
- CN202410593339.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-14
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2044-05-14
AI Technical Summary
Existing machine learning-based methods for identifying financial fraud struggle to effectively capture the internal and inter-modal interactions within multimodal data and cannot effectively differentiate the recognition performance of different modalities. This results in models being overly sensitive to noise or less relevant modalities, leading to insufficient recognition accuracy.
A multimodal attention network-based approach is adopted, which mines the interaction information within and between different modalities through self-attention and guided attention modules, and dynamically adjusts the influence of modal prediction differences in decision-making to construct a multimodal loss function to optimize the model.
It improves the accuracy of financial fraud detection, better combines financial indicators with disclosure text content, automatically assigns importance weights to different modalities, dynamically adjusts modal influence, and enhances the model's recognition performance.
Smart Images

Figure CN118427619B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of big data and machine learning, specifically a method for identifying financial fraud based on multimodal attention networks. Background Technology
[0002] Financial fraud by listed companies aims to deliberately falsify or conceal material facts regarding a company's operating performance and financial condition in order to mislead investors, pursue short-term profits, or protect the interests of management. This behavior severely harms investors' rights and undermines market fairness and transparency. Therefore, effectively identifying financial fraud has become a crucial issue of common concern to both academia and industry.
[0003] With the rapid development of technology, machine learning has achieved remarkable results in many fields and is increasingly being applied to research on identifying financial fraud in listed companies. Currently, innovations in machine learning-based methods for identifying financial fraud mainly focus on two aspects: modal data and method construction. Modal data in machine learning-based methods for identifying financial fraud are primarily divided into two types: financial indicators and disclosure texts. Most existing methods focus on a single type of identification data, especially using only financial indicators to identify fraud, while neglecting the role of using multiple modal data in identifying financial statement fraud.
[0004] Machine learning-based methods for identifying financial fraud generally fall into two categories: traditional machine learning methods and deep learning methods. Traditional machine learning methods, with their strong non-linear fitting and generalization capabilities, are widely used in the field of financial fraud detection. Representative methods include logistic regression, decision trees, random forests, and support vector regression. However, traditional machine learning methods typically require complex feature engineering, relying not only on expert knowledge to extract relevant features for financial fraud identification but also on data dimensionality reduction operations, making it difficult to learn effectively from multimodal data. In contrast, deep learning methods can automatically extract and transform features multiple times to achieve a higher level of abstract representation of data, thus compensating for the shortcomings of traditional machine learning methods. Although deep learning methods are more suitable for multimodal data, existing methods still have certain limitations, struggling to capture information about interactions within and between multiple modalities. Furthermore, existing multimodal data learning strategies often fail to effectively distinguish the recognition performance of different modalities, potentially leading to oversensitivity of the model to noise or less relevant modalities. Summary of the Invention
[0005] The present invention aims to address the shortcomings of the existing technology by proposing a financial fraud identification method based on a multimodal attention network. This method aims to capture the interaction information within and between multiple modalities, while also distinguishing the identification performance of different modalities, thereby improving the accuracy of financial fraud identification.
[0006] To achieve the above-mentioned objectives, the present invention adopts the following technical solution:
[0007] The financial fraud identification method based on a multimodal attention network of the present invention is characterized by the following steps:
[0008] Step 1: Construct a multimodal financial fraud identification dataset D = {(X1,y1),(X2,y2),...,(X...} i ,y i ),...,(X N ,y N )}, where, (X i ,y i Let X be the i-th financial sample. i X represents the multimodal data of the i-th financial sample, and X i ={F i ,T i}, Let d represent the i-th financial modal data, m represent the number of groups of financial modal data, and d represent the id-th financial modal data. f This represents the feature dimension of each set of financial modal data; Let d represent the i-th text modal data, n represent the number of text topics in the text modal data, and d represent the number of topics in the text modal data. t Represents the feature dimension of each text topic; y i It is X i Binary class of genuine counterfeit identification labels;
[0009] Step 2: Construct a multimodal attention network, including: a self-attention module, a guided attention module, and a multimodal combined prediction module.
[0010] Step 2.1, use the self-attention module to apply F i and T i Internal interaction processing is performed, resulting in the i-th internal interaction financial mode F. i,self and the i-th internal interactive text modality T i,self ;
[0011] Step 2.2, the guidance attention block for F i,self and T i,self Cross-modal interaction processing is performed to obtain the text modality F after the i-th financial guidance. i,guide and the text modality T after the i-th text guidance i,guide ;
[0012] Step 2.3, the multimodal prediction module analyzes the text modality F after the i-th financial guidance. i,guide and the text modality T after the i-th text guidance i,guideThe process is performed to obtain the predicted forgery detection result.
[0013] Step 3: Based on the multimodal financial fraud identification dataset D, the multimodal attention network is trained using the gradient descent method, and the multimodal loss is calculated to update the network parameters until the multimodal loss converges or the maximum number of iterations is reached, thereby obtaining the optimal financial fraud identification network after training, which is used to identify fraud in the input financial samples.
[0014] The financial fraud identification method based on multimodal attention networks described in this invention is also characterized in that step 2.1 includes the following steps:
[0015] Step 2.1.1, transfer the i-th financial modal data F i The input is given to the self-attention module, and the i-th internal interaction financial modality containing modal internal interaction information is obtained using equations (1)-(4).
[0016] Q i,ff =F i W i,fq (1)
[0017] K i,ff =F i W i,fk (2)
[0018] V i,ff =F i W i,fv (3)
[0019]
[0020] In equations (1)-(4), Q i,ff Let K be the i-th financial query matrix in the self-attention module. i,ff Let V be the i-th financial key-value matrix in the self-attention module. i,ff Let W be the i-th financial content matrix in the self-attention module. i,fq W i,fk , Let d represent the weight matrix of the i-th financial query, the weight matrix of the i-th financial key value, and the weight matrix of the i-th financial content, respectively. e F represents i,self The feature dimension is denoted by softmax, which represents the activation function.
[0021] Step 2.1.2, transfer the i-th text modal data T iThe input is entered into the self-attention module, and the i-th internal interaction text modality containing modal internal interaction information is obtained using equations (5)-(8).
[0022] Q i,tt =T i W i,tq (5)
[0023] K i,tt =T i W i,tk (6)
[0024] V i,tt =T i W i,tv (7)
[0025]
[0026] In equations (5)-(8), Q i,tt Let K be the i-th text query matrix in the self-attention module. i,tt V is the i-th text key-value matrix in the self-attention module. i,tt Let W be the i-th text content matrix in the self-attention module. i,tq W i,tk , Let represent the weight matrix of the i-th text query, the weight matrix of the i-th text key, and the weight matrix of the i-th text content, respectively.
[0027] Step 2.2 includes the following steps:
[0028] Step 2.2.1, the i-th internal interaction financial mode F i,self and the i-th internal interactive text modality T i,self Input the guidance attention block and use equations (9)-(12) to obtain the i-th internal interaction financial mode F. i,self The i-th internal interactive text modality T under guidance i,self attention distribution Thus, the text modality after the i-th financial guidance is obtained using equation (13).
[0029] Q i,ft =F i,self W i,ftq (9)
[0030] K i,ft =T i,self W i,ftk (10)
[0031] V i,ft=T i,self W i,ftv (11)
[0032]
[0033]
[0034] In equations (9)-(12), Q i,ft Let K be the i-th financial guidance query matrix in the guidance attention module. i,ft V is the i-th financial guidance key-value matrix in the guidance attention module. i,ft This refers to the i-th financial guidance content matrix in the guidance attention module. W i,ftk , These represent the weight matrix for the i-th financial guidance query, the weight matrix for the i-th financial guidance key value, and the weight matrix for the i-th financial guidance content, respectively; d u T represents i,guide Feature dimensions;
[0035] Step 2.2.2, the i-th internal interaction financial mode F i,self and the i-th internal interactive text modality T i,guide Input the guidance attention block and use equations (14)-(17) to obtain the i-th internal interactive text modality T. i,self The i-th internal interaction financial mode F under guidance i,self Attention distribution Thus, the financial modality after the i-th text guidance is obtained using equation (18).
[0036] Q i,tf =T i,self W i,tfq (14)
[0037] K i,tf =F i,self W i,tfk (15)
[0038] V i,tf =F i,self W i,tfv (16)
[0039]
[0040]
[0041] In equations (14)-(17), Q i,tf Let K be the i-th text guidance query matrix in the guidance attention module.i,tf V is the i-th text guidance key-value matrix in the guidance attention module. i,tf For the i-th text guidance content matrix in the guidance attention module, W i,ftk , Let represent the weight matrix of the i-th text guidance query, the weight matrix of the i-th text guidance key value, and the weight matrix of the i-th text guidance content, respectively.
[0042] Step 2.3 includes the following steps:
[0043] Step 2.3.1, use a multilayer perceptron to analyze F. i,guide and F i,guide After processing, the corresponding financial modal prediction probability is obtained. Text modality prediction probability Therefore, the average forecast quality of the financial mode can be obtained using equation (20). Average prediction quality of text modal
[0044]
[0045] In equations (19)-(20), β is a hyperparameter;
[0046] Step 2.3.2, construct X using equation (21) i Corresponding multimodal loss Thus, the predicted forgery detection result is obtained using equation (22).
[0047]
[0048]
[0049] The present invention provides an electronic device, including a memory and a processor, wherein the memory is used to store a program that supports the processor in executing the financial fraud identification method, and the processor is configured to execute the program stored in the memory.
[0050] The present invention discloses a computer-readable storage medium on which a computer program is stored, wherein the computer program is executed by a processor to perform the steps of the financial fraud identification method.
[0051] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0052] 1. This invention combines multiple modal data to identify financial fraud. It also considers the impact of financial indicators and the content of listed companies' disclosure texts on the identification of financial fraud. It believes that financial indicators and disclosure texts are different data formats used to present the company's financial situation. These presentations can reflect the company's financial situation from different perspectives. Therefore, combining financial indicators and disclosure texts can better help identify financial fraud.
[0053] 2. This invention utilizes a self-attention module and a guided attention module to mine interactive information within and between different modalities, respectively. It automatically assigns different importance weights to each set of financial data in the financial modality and each topic text content in the text modality, enabling the adaptive fusion of information within and between different modalities, thereby improving the accuracy of financial fraud detection.
[0054] 3. The present invention fully considers the prediction differences of different modalities in the learning strategy, and can dynamically adjust the influence of each modality in the final decision according to the recognition performance of different modalities. This helps the model to better focus on those modalities that provide useful information, while ignoring those modalities that are noisier or less relevant. Attached Figure Description
[0055] Figure 1 This is a schematic flowchart of the method of the present invention;
[0056] Figure 2 The figure shows the Precision experimental results of the method proposed in this invention;
[0057] Figure 3 The figure shows the experimental results of the Recall method proposed in this invention. Detailed Implementation
[0058] In this embodiment, a financial fraud identification method based on a multimodal attention network utilizes an end-to-end deep learning model, simultaneously considering financial indicators and the content of listed company disclosures, and employs an attention mechanism to mine interaction information within and between different modalities. By considering the predictive differences between different modalities, an accurate identification result is ultimately obtained. Figure 1 As shown, this method for identifying financial fraud includes the following steps:
[0059] Step 1: Construct a multimodal financial fraud identification dataset D = {(X1,y1),(X2,y2),...,(X...} i ,y i ),...,(X N ,y N After performing chapter segmentation, word segmentation, and stop word removal on the text modal data, Doc2Vec is used to vectorize the text modal data, where (X)i ,y i Let X be the i-th financial sample. i Let X represent the multimodal data of the i-th financial sample, and X i ={F i ,T i}, Let d represent the i-th financial modal data, m represent the number of groups of financial modal data, and d represent the id-th financial modal data. f This represents the feature dimension of each set of financial modal data; Let d represent the i-th text modal data, n represent the number of text topics in the text modal data, and d represent the number of topics in the text modal data. t Represents the feature dimension of each text topic; y i It is X i Binary-class labels for identifying genuine fraud; grouping financial modal data and sub-theming text modal data helps to mine interactive information within and between modalities at a certain fine-grained level.
[0060] Step 2: Construct a multimodal attention network, including: a self-attention module, a guided attention module, and a multimodal combined prediction module.
[0061] Step 2.1, use the self-attention module to apply F i and T i Internal interaction processing is performed, resulting in the i-th internal interaction financial mode F. i,self and the i-th internal interactive text modality T i,self The self-attention module is used to mine the interaction information between different groups of financial data within the financial modality and between different thematic text data within the text modality. Based on the mutual guidance between the different groups of financial data and the different thematic text data within the text modality, key information is extracted, and information interaction is performed based on this key information. Ultimately, the internal interactive financial modality F is obtained, representing the interaction information within the meaning modality. i,self and internal interactive text modality T i,self .
[0062] Step 2.1.1, transfer the i-th financial modal data F i The input is processed by the attention module, and the i-th internal interaction financial modality containing modal internal interaction information is obtained using equations (1)-(4).
[0063] Q i,ff =F i W i,fq (1)
[0064] K i,ff =F i W i,fk (2)
[0065] Vi,ff =F i W i,fv (3)
[0066]
[0067] In equations (1)-(4), Q i,ff Let K be the i-th financial query matrix in the self-attention module. i,ff Let V be the i-th financial key-value matrix in the self-attention module. i,ff Let W be the i-th financial content matrix in the self-attention module. i,fq W i,fk , Let d represent the weight matrix of the i-th financial query, the weight matrix of the i-th financial key value, and the weight matrix of the i-th financial content, respectively. e F represents i,self The feature dimension is denoted by softmax, which represents the activation function.
[0068] Step 2.1.2, transfer the i-th text modal data T i The input is processed by the attention module, and the i-th internal interaction text modality containing the modal internal interaction information is obtained using equations (5)-(8).
[0069] Q i,tt =T i W i,tq (5)
[0070] K i,tt =T i W i,tk (6)
[0071] V i,tt =T i W i,tv (7)
[0072]
[0073] In equations (5)-(8), Q i,tt Let K be the i-th text query matrix in the self-attention module. i,tt Let V be the i-th text key-value matrix in the self-attention module. i,tt Let W be the i-th text content matrix in the self-attention module. i,tq W i,tk , Let represent the weight matrix of the i-th text query, the weight matrix of the i-th text key, and the weight matrix of the i-th text content, respectively.
[0074] Step 2.2, guide the attention block to F i,selfand T i,self Internal interaction processing is performed to obtain the financial modality F after the i-th text guidance. i,guide and the text modality T after the i-th financial guidance i,guide The guided attention module aims to uncover the interaction information between financial and textual modal data. Based on the mutual guidance between financial and textual modalities, key information is extracted, and information interaction is performed based on this key information, ultimately yielding the guided financial modality F, which contains the interaction information between semantic modalities. i,guide and guidance text modality T i,guide .
[0075] Step 2.2.1, convert the i-th internal interaction financial mode F i,self and the i-th internal interactive text modality T i,self Input the guidance attention block and use equations (9)-(12) to obtain the i-th internal interaction financial modality F. i,self The i-th internal interactive text modality T under guidance i,self attention distribution This attention distribution focuses on important textual modal information related to the financial modality, thereby obtaining the textual modality after the i-th financial guidance using Equation (13).
[0076] Q i,ft =F i,self W i,ftq (9)
[0077] K i,ft =T i,self W i,ftk (10)
[0078] V i,ft =T i,self W i,ftv (11)
[0079]
[0080]
[0081] In equations (9)-(12), Q i,ft To guide the i-th financial guidance query matrix in the attention module, K i,ft To guide the i-th financial guidance key-value matrix in the attention module, V i,ft To guide the i-th financial guidance content matrix in the attention module, W i,ftk , These represent the weight matrix for the i-th financial guidance query, the weight matrix for the i-th financial guidance key value, and the weight matrix for the i-th financial guidance content, respectively; du T represents i,guide The feature dimensions.
[0082] Step 2.2.2, convert the i-th internal interaction financial mode F i,self and the i-th internal interactive text modality T i,guide Input the guidance attention block and use equations (14)-(17) to obtain the i-th internal interactive text modality T. i,self The i-th internal interaction financial mode F under guidance i,self Attention distribution This attention distribution focuses on important financial modality information related to the text modality, thereby obtaining the financial modality after the i-th text guidance using Equation (18).
[0083] Q i,tf =T i,self W i,tfq (14)
[0084] K i,tf =F i,self W i,tfk (15)
[0085] V i,tf =F i,self W i,tfv (16)
[0086]
[0087]
[0088] In equations (14)-(17), Q i,tf To guide the i-th text-guided query matrix in the attention module, K i,tf To guide the i-th text in the attention module, V i,tf To guide the i-th text guidance content matrix in the attention module, W i,ftk , Let represent the weight matrix of the i-th text guidance query, the weight matrix of the i-th text guidance key value, and the weight matrix of the i-th text guidance content, respectively.
[0089] Step 2.3, the multimodal prediction module processes the text modality F after the i-th financial guidance. i,guide and the text modality T after the i-th text guidance i,guide Perform predictive processing to obtain the predicted forgery detection results.
[0090] Step 2.3.1, use a multilayer perceptron to analyze F. i,guide and Fi,guide After processing, the corresponding financial modal prediction probability is obtained. Text modality prediction probability Therefore, the average forecast quality of the financial mode can be obtained using equation (20). Average prediction quality of text modal Based on the predicted probabilities and average quality of different modes, the model can dynamically adjust the influence of each mode in the final decision according to the prediction quality of different modes. This helps the model to better focus on those modes that provide important information, while ignoring those less important modes.
[0091]
[0092] In equations (19)-(20), β is a hyperparameter used to adjust the influence of different modes on the final loss.
[0093] Step 2.3.2: Learn multimodal data based on prediction probabilities and average prediction quality under different modalities, and construct X using equation (21). i Corresponding multimodal loss Thus, the predicted forgery detection result is obtained using equation (22).
[0094]
[0095]
[0096] Step 3: Based on the multimodal financial fraud identification dataset D, in this example, the gradient descent method is used to train the multimodal attention network, and the multimodal loss is calculated to update the network parameters until the multimodal loss converges or the maximum number of iterations is reached, thereby obtaining the optimal financial fraud identification network after training, which is used to identify fraud in the input financial samples.
[0097] In this embodiment, an electronic device includes a memory and a processor. The memory stores a program that supports the processor in executing the above-described method, and the processor is configured to execute the program stored in the memory.
[0098] In this embodiment, a computer-readable storage medium stores a computer program, which is executed by a processor to perform the steps of the above method.
[0099] The following is an experimental demonstration of the method of the present invention using specific examples. The details are as follows:
[0100] 1. Dataset
[0101] The sample set constructed in this invention is based on the "General List of Violation Information" in the CSMAR database. Companies exhibiting six types of violations (P2501 to P2506) were selected as samples of listed companies' financial statement fraud. A total sample set was constructed at a ratio of 1:11, resulting in an annual sample of 7,560 listed companies from 2015 to 2020. Financial indicators were selected from the financial analysis database within the CSMAR database, totaling 376 financial indicators; company annual reports were sourced from the Shenzhen Stock Exchange, the Shanghai Stock Exchange, and CNINFO.
[0102] 2. Evaluation Indicators
[0103] This invention uses precision and recall as evaluation metrics. Precision represents the proportion of fraudulent samples correctly identified by the model, while recall represents the proportion of all fraudulent samples correctly identified. Both metrics range from 0 to 1; values closer to 1 indicate better model classification performance.
[0104] Wherein, TP indicates that the fraudulent sample was correctly classified as a fraudulent sample, FP indicates that the normal sample was misclassified as a fraudulent sample, and FN indicates that the fraudulent sample was misclassified as a normal sample.
[0105] 3. Experimental Procedure
[0106] To verify the effectiveness of the proposed method, this invention compares it with Logistic Regression (LR), Support Vector Regression (SVM), Decision Tree (DT), Gradient Boosting Decision Tree (GBDT), Multi-Layer Perceptron (MLP), and Long Short-Term Memory (LSTM).
[0107] 4. Experimental Results
[0108] To verify the effectiveness of the method proposed in this invention, experiments were conducted on fraudulent financial statement data from listed companies, and the experimental results of the proposed method and comparative methods were compared. The experimental results are as follows: Figure 2 and Figure 3 As shown.
[0109] Depend on Figure 2 and Figure 3It can be seen that the method proposed in this invention outperforms the other six identification methods in both the Precision and Recall evaluation metrics. Furthermore, it can be seen that compared to using only financial indicators or only disclosure documents, the fraud detection effect using multimodal data is significantly better than using only single-modal data. This result indicates that, for the identification of financial statement fraud in listed companies, the self-attention module and guided attention module proposed in this invention can fully explore the interaction information within and between different modalities, and the multimodal loss function proposed in this invention can fully consider the prediction differences between different modalities, thereby contributing to the improvement of fraud detection effectiveness.
Claims
1. A method for identifying financial fraud based on a multimodal attention network, characterized in that, Includes the following steps: Step 1: Construct a multimodal financial fraud identification dataset ,in, For the i-th financial sample, The multimodal data representing the i-th financial sample, and , This represents the i-th financial modal data. This indicates the number of groups in the financial modal data. This represents the feature dimension of each set of financial modal data; This represents the i-th text modal data. This represents the number of text topics in the text modal data. The feature dimensions representing each text topic; yes Binary class of genuine counterfeit identification labels; Step 2: Construct a multimodal attention network, including: a self-attention module, a guided attention module, and a multimodal combined prediction module. Step 2.1, use the self-attention module to... and Perform internal interaction processing to obtain the i-th internal interaction financial modality. and the i-th internal interactive text modality ; Step 2.2, the guidance attention block pair and Perform cross-modal interaction processing to obtain the text modality after the i-th financial guidance. and the text modality after the i-th text guidance ; Step 2.3, the multimodal prediction module analyzes the text modality after the i-th financial guidance. and the text modality after the i-th text guidance The process is performed to obtain the predicted forgery detection result. ; Step 2.3.1, using a multilayer perceptron to... and After processing, the corresponding financial modal prediction probability is obtained. Text modality prediction probability Thus, the average forecast quality of the financial modality can be obtained using equations (19)-(20). Average prediction quality of text modalities : (19) (20) In equations (19)-(20), For hyperparameters; Step 2.3.2, construct using equation (21) Corresponding multimodal loss Thus, the predicted forgery identification result can be obtained using equation (22). : (21) (22) Step 3, based on the multimodal financial fraud identification dataset The multimodal attention network is trained using gradient descent and multimodal loss is calculated to update the network parameters until the multimodal loss converges or the maximum number of iterations is reached, thereby obtaining the optimal financial fraud detection network after training, which is used to identify fraud in input financial samples.
2. The financial fraud identification method based on multimodal attention network according to claim 1, characterized in that, Step 2.1 includes the following steps: Step 2.1.1, the i-th financial modal data The input is given to the self-attention module, and the i-th internal interaction financial modality containing modal internal interaction information is obtained using equations (1)-(4). : (1) (2) (3) (4) In equations (1)-(4), Let i be the i-th financial query matrix in the self-attention module. Let i be the i-th financial key-value matrix in the self-attention module. Let i be the i-th financial content matrix in the self-attention module. , , Let these represent the weight matrix of the i-th financial query, the weight matrix of the i-th financial key value, and the weight matrix of the i-th financial content, respectively. express The feature dimension is denoted by softmax, which represents the activation function; the superscript T indicates transpose. Step 2.1.2, the i-th text modal data The input is entered into the self-attention module, and the i-th internal interaction text modality containing modal internal interaction information is obtained using equations (5)-(8). : (5) (6) (7) (8) In equations (5)-(8), Let i be the i-th text query matrix in the self-attention module. Let i be the i-th text key-value matrix in the self-attention module. Let i be the i-th text content matrix in the self-attention module. , , Let represent the weight matrix of the i-th text query, the weight matrix of the i-th text key, and the weight matrix of the i-th text content, respectively.
3. The financial fraud identification method based on multimodal attention network according to claim 2, characterized in that, Step 2.2 includes the following steps: Step 2.2.1, the i-th internal interactive financial mode and the i-th internal interactive text modality Input the guidance attention block and use equations (9)-(12) to obtain the i-th internal interaction financial modality. The i-th internal interactive text modality under guidance attention distribution Thus, the text modality after the i-th financial guidance is obtained using equation (13). : (9) (10) (11) (12) (13) In equations (9)-(13), For the i-th financial guidance query matrix in the guidance attention module, Let i be the i-th financial guidance key-value matrix in the guidance attention module. This refers to the i-th financial guidance content matrix in the guidance attention module. , , Let represent the weight matrix of the i-th financial guidance query, the weight matrix of the i-th financial guidance key value, and the weight matrix of the i-th financial guidance content, respectively. express Feature dimensions; Step 2.2.2, the i-th internal interaction financial mode and the i-th internal interactive text modality Input the guidance attention block and use equations (14)-(17) to obtain the i-th internal interactive text modality. The i-th internal interaction financial modality under guidance Attention distribution Thus, the financial modality after the i-th text guidance is obtained using equation (18). : (14) (15) (16) (17) (18) In equations (14)-(18), For the i-th text guidance query matrix in the guidance attention module, Let i be the text guidance key-value matrix in the guidance attention module. For the i-th text guidance content matrix in the guidance attention module, , , Let represent the weight matrix of the i-th text guidance query, the weight matrix of the i-th text guidance key value, and the weight matrix of the i-th text guidance content, respectively.
4. An electronic device, comprising a memory and a processor, characterized in that, The memory is used to store a program that supports the processor in executing any of the financial fraud identification methods of claims 1-3, and the processor is configured to execute the program stored in the memory.
5. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is run by the processor, it performs the steps of any of the financial fraud identification methods described in claims 1-3.
Citation Information
Patent Citations
Method and device for detecting abnormal data and computer readable storage medium
CN112016934A
Multi-modal fusion emotion recognition system and method based on multi-task learning and attention mechanism and experimental evaluation method
CN113420807A
