Payment safety monitoring method and system based on multi-modal data

Abstract

The invention discloses a payment security monitoring method and system based on multi-modal data. The method comprises the following steps: acquiring physiological data and transaction data of a monitored object; extracting multi-modal features corresponding to the physiological data and the transaction data according to a geometric mean algorithm and a graph convolutional network; the multi-modal features comprise a geometric mean feature matrix and graph network features; on the basis of an LSTM neural network, according to the multi-modal features, predicting a health trend corresponding to the monitored object; and according to the health trend and the transaction data, determining a transaction security condition corresponding to the monitored object, and giving an alarm or intervening when the transaction security condition is abnormal. Therefore, the method can achieve the intelligent recognition of the potential association between the health state of the user and the transaction behavior, effectively improves the prediction accuracy and response timeliness of the financial transaction risk, and improves the safety and intervention capability of the transaction process.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a payment security monitoring method and system based on multimodal data. Background Art

[0002] Existing technologies often rely on static analysis of transaction behavior characteristics, failing to fully capture the underlying factors behind user behavior, such as physiological or health status. This results in limited ability to identify complex or hidden abnormal transaction behaviors. Furthermore, existing technologies often overlook the potential correlation between user physiological data and transaction behavior and lack effective cross-modal feature fusion methods. This results in insufficient accuracy and foresight in transaction risk assessment results, making it impossible to promptly respond to potential transaction risks arising from sudden health changes, impacting the security of the overall financial system and user experience. Clearly, existing technologies suffer from shortcomings that urgently need to be addressed. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a payment security monitoring method and system based on multimodal data, which can realize the intelligent identification of the potential correlation between user health status and transaction behavior, effectively improve the prediction accuracy and response timeliness of financial transaction risks, and enhance the security and intervention capabilities of the transaction process.

[0004] In order to solve the above technical problems, the first aspect of the present invention discloses a payment security monitoring method based on multimodal data, the method comprising: Obtaining physiological data and transaction data of monitored subjects; Extracting multimodal features corresponding to the physiological data and the transaction data based on a geometric mean algorithm and a graph convolutional network; the multimodal features include a geometric mean feature matrix and graph network features; Based on the LSTM neural network, predict the health trend of the monitored object according to the multimodal features; Based on the health trend and the transaction data, the transaction security situation corresponding to the monitored object is judged, and an alarm or intervention is issued when there is an abnormality in the transaction security situation.

[0005] As an optional embodiment, in the first aspect of the present invention, the physiological data includes at least one of heart rate data, blood oxygen data, skin electricity data and body temperature data.

[0006] As an optional implementation, in the first aspect of the present invention, the transaction data includes at least one of the transaction object, transaction amount, transaction location and transaction time.

[0007] As an optional embodiment, in the first aspect of the present invention, extracting multimodal features corresponding to the physiological data and the transaction data based on a geometric mean algorithm and a graph convolutional network includes: Calculating the geometric mean of multiple data corresponding to any time period of the physiological data to obtain a horizontal geometric mean feature corresponding to the physiological data; Calculating the geometric mean of data values ​​of the same type of data in the physiological data at multiple time points to obtain a longitudinal geometric mean feature corresponding to the physiological data; Combining the transverse geometric mean feature and the longitudinal geometric mean feature into a geometric mean feature matrix; constructing a graph network corresponding to the physiological data and the transaction data based on the correlation between the physiological data and the transaction data; The graph network is input into a trained graph convolutional network to obtain graph network features corresponding to the graph network; the graph convolutional network is trained by a training data set including multiple training graph networks and corresponding feature annotations.

[0008] As an optional embodiment, in the first aspect of the present invention, constructing a graph network corresponding to the physiological data and the transaction data based on the correlation between the physiological data and the transaction data includes: Each preset time period is regarded as a graph node, and the physiological data and the transaction data related to the time period are determined as graph node representation data corresponding to the graph node; For any two adjacent graph nodes, calculate an adjacency parameter between the graph node representation data corresponding to the two graph nodes; the adjacency parameter is a weighted sum of a physiological association parameter and a transaction association parameter; the physiological association parameter is the similarity between the physiological data corresponding to the two graph nodes; the transaction association parameter is the similarity between the transaction data corresponding to the two graph nodes; All the graph nodes and the corresponding adjacency parameters therebetween are determined as a graph network corresponding to the physiological data and the transaction data.

[0009] As an optional embodiment, in the first aspect of the present invention, the LSTM neural network-based, predicting the health trend corresponding to the monitored object according to the multimodal features includes: Combining the geometric mean feature matrices corresponding to the monitored object at multiple time points from early to late based on the time points to obtain a feature matrix sequence; Combining the graph network features corresponding to the monitored object at multiple time points from early to late based on the time points to obtain a graph network feature sequence; Normalizing and time-point matching the feature matrix sequence and the graph structure feature sequence to obtain a fused feature sequence; The fused feature sequence is input into a trained LSTM neural network to obtain a health trend corresponding to the monitored object; the health trend includes health status parameters of the monitored object at at least one future time point.

[0010] As an optional embodiment, in the first aspect of the present invention, the LSTM neural network is trained by a training data set including multiple training health condition sequences and corresponding fusion feature annotations, and the forgetting gate parameter of the LSTM neural network is set to be inversely proportional to the QD value between the fusion feature annotation of the current input and the fusion feature annotation of the previous input; the QD value is the logarithm with the golden section constant as the base corresponding to the ratio of the values ​​of the two fusion feature annotations; the fusion feature annotation is a geometric feature matrix or a graph network feature.

[0011] As an optional embodiment, in the first aspect of the present invention, determining the transaction security status corresponding to the monitored object based on the health trend and the transaction data includes: Calculate the difference between the health condition parameter at a future time point in the health trend and the health reference parameter to obtain the health characterization parameter Inputting the transaction data into a trained transaction anomaly recognition neural network to obtain transaction anomaly recognition parameters; Determine whether the health characterization parameter is greater than a first parameter threshold, and obtain a first determination result; Determining whether the transaction anomaly identification parameter is greater than a second parameter threshold, and obtaining a second determination result; When both the first judgment result and the second judgment result are yes, it is determined that there is an abnormality in the transaction security situation corresponding to the monitored object.

[0012] A second aspect of an embodiment of the present invention discloses a payment security monitoring system based on multimodal data, the system comprising: An acquisition module, used to obtain physiological data and transaction data of the monitored object; an extraction module, configured to extract multimodal features corresponding to the physiological data and the transaction data based on a geometric mean algorithm and a graph convolutional network; the multimodal features comprising a geometric mean feature matrix and graph network features; A prediction module, configured to predict the health trend of the monitored subject based on the multimodal features based on an LSTM neural network; The judgment module is used to judge the transaction security situation corresponding to the monitored object based on the health trend and the transaction data, and to issue an alarm or intervene when there is an abnormality in the transaction security situation.

[0013] As an optional embodiment, in the second aspect of the present invention, the physiological data includes at least one of heart rate data, blood oxygen data, skin electricity data and body temperature data.

[0014] As an optional implementation, in the second aspect of the present invention, the transaction data includes at least one of the transaction object, transaction amount, transaction location and transaction time.

[0015] As an optional embodiment, in the second aspect of the present invention, the specific manner in which the extraction module extracts the multimodal features corresponding to the physiological data and the transaction data based on the geometric mean algorithm and the graph convolutional network includes: Calculating the geometric mean of multiple data corresponding to any time period of the physiological data to obtain a horizontal geometric mean feature corresponding to the physiological data; Calculating the geometric mean of data values ​​of the same type of data in the physiological data at multiple time points to obtain a longitudinal geometric mean feature corresponding to the physiological data; Combining the transverse geometric mean feature and the longitudinal geometric mean feature into a geometric mean feature matrix; constructing a graph network corresponding to the physiological data and the transaction data based on the correlation between the physiological data and the transaction data; The graph network is input into a trained graph convolutional network to obtain graph network features corresponding to the graph network; the graph convolutional network is trained by a training data set including multiple training graph networks and corresponding feature annotations.

[0016] As an optional embodiment, in the second aspect of the present invention, the specific manner in which the extraction module constructs a graph network corresponding to the physiological data and the transaction data based on the correlation between the physiological data and the transaction data includes: Each preset time period is regarded as a graph node, and the physiological data and the transaction data related to the time period are determined as graph node representation data corresponding to the graph node; For any two adjacent graph nodes, calculate an adjacency parameter between the graph node representation data corresponding to the two graph nodes; the adjacency parameter is a weighted sum of a physiological association parameter and a transaction association parameter; the physiological association parameter is the similarity between the physiological data corresponding to the two graph nodes; the transaction association parameter is the similarity between the transaction data corresponding to the two graph nodes; All the graph nodes and the corresponding adjacency parameters therebetween are determined as a graph network corresponding to the physiological data and the transaction data.

[0017] As an optional embodiment, in the second aspect of the present invention, the prediction module is based on an LSTM neural network, and according to the multimodal features, the specific manner of predicting the health trend corresponding to the monitored object includes: Combining the geometric mean feature matrices corresponding to the monitored object at multiple time points from early to late based on the time points to obtain a feature matrix sequence; Combining the graph network features corresponding to the monitored object at multiple time points from early to late based on the time points to obtain a graph network feature sequence; Normalizing and time-point matching the feature matrix sequence and the graph structure feature sequence to obtain a fused feature sequence; The fused feature sequence is input into a trained LSTM neural network to obtain a health trend corresponding to the monitored object; the health trend includes health status parameters of the monitored object at at least one future time point.

[0018] As an optional embodiment, in the second aspect of the present invention, the LSTM neural network is trained by a training data set including multiple training health condition sequences and corresponding fusion feature annotations, and the forgetting gate parameter of the LSTM neural network is set to be inversely proportional to the QD value between the fusion feature annotation of the current input and the fusion feature annotation of the previous input; the QD value is the logarithm with the golden section constant as the base corresponding to the ratio of the values ​​of the two fusion feature annotations; the fusion feature annotation is a geometric feature matrix or a graph network feature.

[0019] As an optional embodiment, in the second aspect of the present invention, the specific manner in which the judgment module judges the transaction security status corresponding to the monitored object based on the health trend and the transaction data includes: Calculate the difference between the health condition parameter at a future time point in the health trend and the health reference parameter to obtain the health characterization parameter Inputting the transaction data into a trained transaction anomaly recognition neural network to obtain transaction anomaly recognition parameters; Determine whether the health characterization parameter is greater than a first parameter threshold, and obtain a first determination result; Determining whether the transaction anomaly identification parameter is greater than a second parameter threshold, and obtaining a second determination result; When both the first judgment result and the second judgment result are yes, it is determined that there is an abnormality in the transaction security situation corresponding to the monitored object.

[0020] A third aspect of the present invention discloses another payment security monitoring system based on multimodal data, the system comprising: a memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute part or all of the steps in the payment security monitoring method based on multimodal data disclosed in the first aspect of the present invention.

[0021] The fourth aspect of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute some or all of the steps in the payment security monitoring method based on multimodal data disclosed in the first aspect of the present invention.

[0022] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: The present invention jointly extracts multimodal features from the physiological data and transaction data of the monitored subjects through the geometric mean algorithm and graph convolutional network, dynamically predicts the health trends of the monitored subjects using the LSTM neural network, and judges the transaction security status by combining the health trends and transaction data. This enables intelligent identification of the potential correlation between the user's health status and transaction behavior, effectively improving the prediction accuracy and response timeliness of financial transaction risks, and enhancing the security and intervention capabilities of the transaction process. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 This is a flow chart of a payment security monitoring method based on multimodal data disclosed in an embodiment of the present invention.

[0025] Figure 2 It is a structural diagram of a payment security monitoring system based on multimodal data disclosed in an embodiment of the present invention.

[0026] Figure 3 This is a structural diagram of another payment security monitoring system based on multimodal data disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0028] The terms "first," "second," and so on, in the description and claims of the present invention and the accompanying drawings are used to distinguish between different objects, not to describe a specific order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device comprising a series of steps or elements is not limited to the listed steps or elements but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or device.

[0029] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute a separate or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0030] The present invention discloses a payment security monitoring method and system based on multimodal data. This method uses a geometric mean algorithm and a graph convolutional network to jointly extract multimodal features from the physiological data and transaction data of the monitored subject. It then uses an LSTM neural network to dynamically predict the health trends of the monitored subject and combines these health trends with transaction data to determine the transaction security status. This method can intelligently identify the potential correlation between a user's health status and transaction behavior, effectively improving the accuracy of financial transaction risk prediction and the timeliness of response, and enhancing the security and intervention capabilities of the transaction process. These are described in detail below.

[0031] Example 1 See also Figure 1 , Figure 1 This is a flow chart of a payment security monitoring method based on multimodal data disclosed in an embodiment of the present invention. Figure 1 The described payment security monitoring method based on multimodal data can be applied to a data processing system / data processing device / data processing server (wherein the server includes a local processing server or a cloud processing server). Figure 1 As shown, the payment security monitoring method based on multimodal data may include the following operations: 101. Obtain physiological data and transaction data of the monitored object.

[0032] 102. Based on the geometric mean algorithm and graph convolutional network, multimodal features corresponding to physiological data and transaction data are extracted. Optionally, the multimodal features include geometric mean feature matrix and graph network features.

[0033] 103. Based on LSTM neural network, the health trend of the monitored object is predicted according to multimodal features. 104. Based on health trends and transaction data, determine the transaction security status of the monitored object, and issue an alarm or intervene when there are abnormalities in the transaction security status.

[0034] It can be seen that the above-mentioned embodiment of the invention jointly extracts multimodal features from the physiological data and transaction data of the monitored object through the geometric mean algorithm and the graph convolutional network, uses the LSTM neural network to dynamically predict the health trend of the monitored object, and combines the health trend and transaction data to judge its transaction security, thereby realizing intelligent identification of the potential correlation between the user's health status and transaction behavior, effectively improving the prediction accuracy and response timeliness of financial transaction risks, and enhancing the security and intervention capabilities of the transaction process.

[0035] As an optional embodiment, in the above steps, the physiological data includes at least one of heart rate data, blood oxygen data, skin electricity data and body temperature data.

[0036] It can be seen that through the above optional embodiments, the content of physiological data is limited to comprehensively characterize the physiological characteristics of the monitored object, assist in realizing the intelligent identification of potential correlations between user health status and transaction behavior, effectively improve the prediction accuracy and response timeliness of financial transaction risks, and enhance the security and intervention capabilities of the transaction process.

[0037] As an optional embodiment, in the above steps, the transaction data includes at least one of the transaction object, transaction amount, transaction location and transaction time.

[0038] It can be seen that through the above optional embodiments, the content of transaction data is limited to comprehensively characterize the transaction characteristics of the monitored object, assist in realizing the intelligent identification of potential correlations between user health status and transaction behavior, effectively improve the prediction accuracy and response timeliness of financial transaction risks, and enhance the security and intervention capabilities of the transaction process.

[0039] As an optional embodiment, in the above steps, extracting multimodal features corresponding to physiological data and transaction data based on the geometric mean algorithm and graph convolutional network includes: Calculating the geometric mean of multiple data corresponding to any time period in the physiological data to obtain a horizontal geometric mean feature corresponding to the physiological data; Calculating the geometric mean of data values ​​of the same type of data in the physiological data at multiple time points to obtain a longitudinal geometric mean feature corresponding to the physiological data; Combine the horizontal geometric mean features and the vertical geometric mean features into a geometric mean feature matrix; Based on the correlation between physiological data and transaction data, a graph network corresponding to physiological data and transaction data is constructed; The graph network is input into the trained graph convolutional network to obtain the graph network features corresponding to the graph network; the graph convolutional network is trained by a training data set including multiple training graph networks and corresponding feature annotations.

[0040] It can be seen that through the above optional embodiments, by calculating the geometric mean of different dimensions and time periods of physiological data, extracting horizontal and vertical stability features respectively, and combining them into a geometric mean feature matrix, further constructing a graph structure based on the correlation between physiological data and transaction data and introducing a graph convolutional network for feature extraction, it is possible to effectively mine and integrate deep correlations between multimodal data, improve the mapping ability of health status changes in transaction security assessment, thereby enhancing the accuracy and timeliness of health-driven transaction abnormality risk identification, and significantly improving the system's early warning effect and practical value for complex abnormal transaction behaviors.

[0041] As an optional embodiment, in the above step, constructing a graph network corresponding to the physiological data and the transaction data based on the correlation between the physiological data and the transaction data includes: Each preset time period is regarded as a graph node, and the physiological data and transaction data related to the time period are determined as graph node representation data corresponding to the graph node; For any two adjacent graph nodes, calculate the adjacency parameter between the graph node representation data corresponding to the two graph nodes; optionally, the adjacency parameter is a weighted sum of a physiological association parameter and a transaction association parameter; the physiological association parameter is the similarity between the physiological data corresponding to the two graph nodes; and the transaction association parameter is the similarity between the transaction data corresponding to the two graph nodes; The corresponding adjacency parameters between all graph nodes are determined as the graph network corresponding to physiological data and transaction data.

[0042] It can be seen that through the above optional embodiments, by taking each preset time period as a graph node in the graph structure, and constructing graph node representation data with physiological data and transaction data within the time period, and then calculating the physiological similarity and transaction similarity between adjacent time periods, and constructing the adjacency parameters in the graph structure through weighted fusion, a graph network that integrates time series information and multimodal association features is constructed, which can effectively express the dynamic connection between physiological and transaction behaviors while ensuring time continuity, enhance the expressive ability of the graph structure and the model's ability to characterize abnormal evolution trends, and improve the accuracy and robustness of subsequent risk prediction and health assessment based on graph neural networks.

[0043] As an optional embodiment, in the above steps, based on the LSTM neural network and according to the multimodal features, predicting the health trend corresponding to the monitored object includes: The geometric mean feature matrices corresponding to the monitored objects at multiple time points are combined from early to late based on the time points to obtain a feature matrix sequence; The graph network features corresponding to the monitored object at multiple time points are combined from early to late based on the time points to obtain a graph network feature sequence; Normalize the feature matrix sequence and the graph structure feature sequence and combine them with time points to obtain a fused feature sequence; The fused feature sequence is input into the trained LSTM neural network to obtain the health trend corresponding to the monitored object; the health trend includes the health status parameters of the monitored object at at least one future time point.

[0044] It can be seen that through the above optional embodiments, by combining the geometric mean feature matrix and graph network features corresponding to multiple time points in chronological order to form a feature matrix sequence and a graph network feature sequence, and fusing them into a unified fusion feature sequence after normalization and time alignment, and then inputting them into the trained LSTM neural network to realize the prediction of health trends, it can make full use of the collaborative information between time series characteristics and multimodal features, improve the dynamic modeling ability of the health evolution process of the monitored object, enhance the accuracy and foresight of future health status prediction, and provide efficient and reliable technical support for personalized health management and intelligent early warning.

[0045] As an optional embodiment, in the above steps, the LSTM neural network is trained by a training data set including multiple training health condition sequences and corresponding fusion feature annotations, and the forget gate parameter of the LSTM neural network is set to be inversely proportional to the QD value between the fusion feature annotation of the current input and the fusion feature annotation of the previous input; the QD value is the logarithm with the golden section constant as the base corresponding to the ratio of the values ​​of the two fusion feature annotations; the fusion feature annotation is a geometric feature matrix or a graph network feature.

[0046] It can be seen that through the above optional embodiments, by introducing the QD value based on the change amplitude between the fused feature annotations to control the forgetting gate parameter when training the LSTM neural network, and making the forgetting gate parameter inversely proportional to the QD value between the fused feature annotations, it is possible to achieve sensitive retention of feature changes at critical moments and appropriate forgetting of features in the stable stage, effectively improving the neural network's modeling ability for the dynamic change process of health status, enhancing the model's adaptability and robustness in complex and changeable health trend predictions, and significantly improving the prediction accuracy and stability of future health status parameters of the monitored objects.

[0047] As an optional embodiment, in the above steps, judging the transaction security status of the monitored object according to the health trend and transaction data includes: Calculate the difference between the health condition parameters at future time points in the health trend and the health reference parameters to obtain the health representation parameters Input the transaction data into the trained transaction anomaly recognition neural network to obtain transaction anomaly recognition parameters; Determine whether the health characterization parameter is greater than a first parameter threshold, and obtain a first determination result; Determining whether the transaction anomaly identification parameter is greater than a second parameter threshold, and obtaining a second determination result; When both the first judgment result and the second judgment result are yes, it is determined that there is an abnormality in the transaction security situation corresponding to the monitored object.

[0048] It can be seen that through the above optional embodiments, by calculating the difference between the health status parameters at a future time point and the reference health parameters, the degree of health deviation of the monitored object is quantified, and combined with the transaction data, it is input into the trained transaction anomaly recognition neural network to obtain the transaction anomaly recognition parameters. On this basis, the joint anomalies of the health and transaction dimensions are identified through a dual judgment mechanism, thereby achieving a comprehensive assessment and accurate identification of the transaction security status of the monitored object, significantly improving the accuracy and reliability of anomaly identification, and ensuring the timeliness of risk warnings and the security of transaction behaviors.

[0049] Example 2 See also Figure 2 , Figure 2 This is a schematic diagram of the structure of a payment security monitoring system based on multimodal data disclosed in an embodiment of the present invention. Figure 2 The described payment security monitoring system based on multimodal data can be applied to a data processing system / data processing device / data processing server (wherein the server includes a local processing server or a cloud processing server). Figure 2 As shown, the payment security monitoring system based on multimodal data may include: The acquisition module 201 is used to acquire the physiological data and transaction data of the monitored object.

[0050] The extraction module 202 is used to extract multimodal features corresponding to physiological data and transaction data based on the geometric mean algorithm and graph convolutional network. Optionally, the multimodal features include geometric mean feature matrix and graph network features.

[0051] The prediction module 203 is used to predict the health trend corresponding to the monitored object based on the LSTM neural network and multimodal features. The judgment module 204 is used to judge the transaction security status corresponding to the monitored object based on the health trend and transaction data, and to issue an alarm or intervene when there is an abnormality in the transaction security status.

[0052] It can be seen that the above-mentioned embodiment of the invention jointly extracts multimodal features from the physiological data and transaction data of the monitored object through the geometric mean algorithm and the graph convolutional network, uses the LSTM neural network to dynamically predict the health trend of the monitored object, and combines the health trend and transaction data to judge its transaction security, thereby realizing intelligent identification of the potential correlation between the user's health status and transaction behavior, effectively improving the prediction accuracy and response timeliness of financial transaction risks, and enhancing the security and intervention capabilities of the transaction process.

[0053] As an optional embodiment, the physiological data includes at least one of heart rate data, blood oxygen data, skin electricity data and body temperature data.

[0054] It can be seen that through the above optional embodiments, the content of physiological data is limited to comprehensively characterize the physiological characteristics of the monitored object, assist in realizing the intelligent identification of potential correlations between user health status and transaction behavior, effectively improve the prediction accuracy and response timeliness of financial transaction risks, and enhance the security and intervention capabilities of the transaction process.

[0055] As an optional embodiment, the transaction data includes at least one of the transaction object, transaction amount, transaction location and transaction time.

[0056] It can be seen that through the above optional embodiments, the content of transaction data is limited to comprehensively characterize the transaction characteristics of the monitored object, assist in realizing the intelligent identification of potential correlations between user health status and transaction behavior, effectively improve the prediction accuracy and response timeliness of financial transaction risks, and enhance the security and intervention capabilities of the transaction process.

[0057] As an optional embodiment, the specific method of extracting multimodal features corresponding to physiological data and transaction data by the extraction module based on the geometric mean algorithm and graph convolutional network includes: Calculating the geometric mean of multiple data corresponding to any time period in the physiological data to obtain a horizontal geometric mean feature corresponding to the physiological data; Calculating the geometric mean of data values ​​of the same type of data in the physiological data at multiple time points to obtain a longitudinal geometric mean feature corresponding to the physiological data; Combine the horizontal geometric mean features and the vertical geometric mean features into a geometric mean feature matrix; Based on the correlation between physiological data and transaction data, a graph network corresponding to physiological data and transaction data is constructed; The graph network is input into the trained graph convolutional network to obtain the graph network features corresponding to the graph network; the graph convolutional network is trained by a training data set including multiple training graph networks and corresponding feature annotations.

[0058] It can be seen that through the above optional embodiments, by calculating the geometric mean of different dimensions and time periods of physiological data, extracting horizontal and vertical stability features respectively, and combining them into a geometric mean feature matrix, further constructing a graph structure based on the correlation between physiological data and transaction data and introducing a graph convolutional network for feature extraction, it is possible to effectively mine and integrate deep correlations between multimodal data, improve the mapping ability of health status changes in transaction security assessment, thereby enhancing the accuracy and timeliness of health-driven transaction abnormality risk identification, and significantly improving the system's early warning effect and practical value for complex abnormal transaction behaviors.

[0059] As an optional embodiment, the specific manner in which the extraction module constructs a graph network corresponding to the physiological data and the transaction data based on the correlation between the physiological data and the transaction data includes: Each preset time period is regarded as a graph node, and the physiological data and transaction data related to the time period are determined as graph node representation data corresponding to the graph node; For any two adjacent graph nodes, calculate the adjacency parameter between the graph node representation data corresponding to the two graph nodes; optionally, the adjacency parameter is a weighted sum of a physiological association parameter and a transaction association parameter; the physiological association parameter is the similarity between the physiological data corresponding to the two graph nodes; and the transaction association parameter is the similarity between the transaction data corresponding to the two graph nodes; The corresponding adjacency parameters between all graph nodes are determined as the graph network corresponding to physiological data and transaction data.

[0060] It can be seen that through the above optional embodiments, by taking each preset time period as a graph node in the graph structure, and constructing graph node representation data with physiological data and transaction data within the time period, and then calculating the physiological similarity and transaction similarity between adjacent time periods, and constructing the adjacency parameters in the graph structure through weighted fusion, a graph network that integrates time series information and multimodal association features is constructed, which can effectively express the dynamic connection between physiological and transaction behaviors while ensuring time continuity, enhance the expressive ability of the graph structure and the model's ability to characterize abnormal evolution trends, and improve the accuracy and robustness of subsequent risk prediction and health assessment based on graph neural networks.

[0061] As an optional embodiment, the prediction module is based on an LSTM neural network and predicts the specific method of health trends corresponding to the monitored object according to multimodal features, including: The geometric mean feature matrices corresponding to the monitored objects at multiple time points are combined from early to late based on the time points to obtain a feature matrix sequence; The graph network features corresponding to the monitored object at multiple time points are combined from early to late based on the time points to obtain a graph network feature sequence; Normalize the feature matrix sequence and the graph structure feature sequence and combine them with time points to obtain a fused feature sequence; The fused feature sequence is input into the trained LSTM neural network to obtain the health trend corresponding to the monitored object; the health trend includes the health status parameters of the monitored object at at least one future time point.

[0062] It can be seen that through the above optional embodiments, by combining the geometric mean feature matrix and graph network features corresponding to multiple time points in chronological order to form a feature matrix sequence and a graph network feature sequence, and fusing them into a unified fusion feature sequence after normalization and time alignment, and then inputting them into the trained LSTM neural network to realize the prediction of health trends, it can make full use of the collaborative information between time series characteristics and multimodal features, improve the dynamic modeling ability of the health evolution process of the monitored object, enhance the accuracy and foresight of future health status prediction, and provide efficient and reliable technical support for personalized health management and intelligent early warning.

[0063] As an optional embodiment, the LSTM neural network is trained using a training data set including multiple training health condition sequences and corresponding fusion feature annotations. The forget gate parameter of the LSTM neural network is set to be inversely proportional to the QD value between the fusion feature annotation of the current input and the fusion feature annotation of the previous input; the QD value is the logarithm of the ratio of the values ​​of the two fusion feature annotations with the golden section constant as the base; the fusion feature annotation is a geometric feature matrix or a graph network feature.

[0064] It can be seen that through the above optional embodiments, by introducing the QD value based on the change amplitude between the fused feature annotations to control the forgetting gate parameter when training the LSTM neural network, and making the forgetting gate parameter inversely proportional to the QD value between the fused feature annotations, it is possible to achieve sensitive retention of feature changes at critical moments and appropriate forgetting of features in the stable stage, effectively improving the neural network's modeling ability for the dynamic change process of health status, enhancing the model's adaptability and robustness in complex and changeable health trend predictions, and significantly improving the prediction accuracy and stability of future health status parameters of the monitored objects.

[0065] As an optional embodiment, the specific manner in which the judgment module judges the transaction security status corresponding to the monitored object based on the health trend and transaction data includes: Calculate the difference between the health condition parameters at future time points in the health trend and the health reference parameters to obtain the health representation parameters Input the transaction data into the trained transaction anomaly recognition neural network to obtain transaction anomaly recognition parameters; Determine whether the health characterization parameter is greater than a first parameter threshold, and obtain a first determination result; Determining whether the transaction anomaly identification parameter is greater than a second parameter threshold, and obtaining a second determination result; When both the first judgment result and the second judgment result are yes, it is determined that there is an abnormality in the transaction security situation corresponding to the monitored object.

[0066] It can be seen that through the above optional embodiments, by calculating the difference between the health status parameters at a future time point and the reference health parameters, the degree of health deviation of the monitored object is quantified, and combined with the transaction data, it is input into the trained transaction anomaly recognition neural network to obtain the transaction anomaly recognition parameters. On this basis, the joint anomalies of the health and transaction dimensions are identified through a dual judgment mechanism, thereby achieving a comprehensive assessment and accurate identification of the transaction security status of the monitored object, significantly improving the accuracy and reliability of anomaly identification, and ensuring the timeliness of risk warnings and the security of transaction behaviors.

[0067] Example 3 See also Figure 3 , Figure 3 This is another payment security monitoring system based on multimodal data disclosed in an embodiment of the present invention. Figure 3 The described payment security monitoring system based on multimodal data is applied to a data processing system / data processing device / data processing server (wherein the server includes a local processing server or a cloud processing server). Figure 3 As shown, the payment security monitoring system based on multimodal data may include: A memory 301 storing executable program code; a processor 302 coupled to the memory 301; The processor 302 calls the executable program code stored in the memory 301 to execute the steps of the payment security monitoring method based on multimodal data described in the first embodiment.

[0068] Example 4 An embodiment of the present invention discloses a computer-readable storage medium storing a computer program for electronic data exchange, wherein the computer program enables a computer to execute the steps of the payment security monitoring method based on multimodal data described in the first embodiment.

[0069] Example 5 An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute the steps of the payment security monitoring method based on multimodal data described in Example 1.

[0070] The foregoing description of specific embodiments of the present disclosure is intended to illustrate a method for performing a multi-tasking process. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0071] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0072] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0073] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0074] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0075] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0076] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0077] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0078] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0079] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0080] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0081] This specification may be described in the general context of computer-executable instructions, such as program modules, executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media, including storage devices.

[0082] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0083] Finally, it should be noted that the payment security monitoring method and system based on multimodal data disclosed in the embodiments of the present invention are only preferred embodiments of the present invention, and are only used to illustrate the technical solutions of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A payment security monitoring method based on multimodal data, characterized in that: The method comprises: Obtaining physiological data and transaction data of monitored subjects; Extracting multimodal features corresponding to the physiological data and the transaction data based on a geometric mean algorithm and a graph convolutional network; the multimodal features include a geometric mean feature matrix and graph network features; Based on the LSTM neural network, predict the health trend of the monitored object according to the multimodal features; Based on the health trend and the transaction data, the transaction security situation corresponding to the monitored object is judged, and an alarm or intervention is issued when there is an abnormality in the transaction security situation.

2. The payment security monitoring method based on multimodal data according to claim 1, characterized in that: The physiological data includes at least one of heart rate data, blood oxygen data, skin electricity data and body temperature data.

3. The payment security monitoring method based on multimodal data according to claim 1, characterized in that: The transaction data includes at least one of the transaction object, transaction amount, transaction location and transaction time.

4. The payment security monitoring method based on multimodal data according to claim 1, characterized in that: The extracting of multimodal features corresponding to the physiological data and the transaction data according to the geometric mean algorithm and the graph convolutional network includes: Calculating the geometric mean of multiple data corresponding to any time period of the physiological data to obtain a horizontal geometric mean feature corresponding to the physiological data; Calculating the geometric mean of data values ​​of the same type of data in the physiological data at multiple time points to obtain a longitudinal geometric mean feature corresponding to the physiological data; Combining the transverse geometric mean feature and the longitudinal geometric mean feature into a geometric mean feature matrix; constructing a graph network corresponding to the physiological data and the transaction data based on the correlation between the physiological data and the transaction data; The graph network is input into a trained graph convolutional network to obtain graph network features corresponding to the graph network; the graph convolutional network is trained by a training data set including multiple training graph networks and corresponding feature annotations.

5. The payment security monitoring method based on multimodal data according to claim 4 is characterized in that: The constructing a graph network corresponding to the physiological data and the transaction data based on the correlation between the physiological data and the transaction data includes: Each preset time period is regarded as a graph node, and the physiological data and the transaction data related to the time period are determined as graph node representation data corresponding to the graph node; For any two adjacent graph nodes, calculate an adjacency parameter between the graph node representation data corresponding to the two graph nodes; the adjacency parameter is a weighted sum of a physiological association parameter and a transaction association parameter; the physiological association parameter is the similarity between the physiological data corresponding to the two graph nodes; the transaction association parameter is the similarity between the transaction data corresponding to the two graph nodes; All the graph nodes and the corresponding adjacency parameters therebetween are determined as a graph network corresponding to the physiological data and the transaction data.

6. The payment security monitoring method based on multimodal data according to claim 1, characterized in that: The LSTM neural network-based method predicts the health trend of the monitored object according to the multimodal features, including: Combining the geometric mean feature matrices corresponding to the monitored object at multiple time points from early to late based on the time points to obtain a feature matrix sequence; Combining the graph network features corresponding to the monitored object at multiple time points from early to late based on the time points to obtain a graph network feature sequence; Normalizing and time-point matching the feature matrix sequence and the graph structure feature sequence to obtain a fused feature sequence; The fused feature sequence is input into a trained LSTM neural network to obtain a health trend corresponding to the monitored object; the health trend includes health status parameters of the monitored object at at least one future time point.

7. The payment security monitoring method based on multimodal data according to claim 6, characterized in that: The LSTM neural network is trained using a training data set comprising multiple training health condition sequences and corresponding fusion feature annotations. The forget gate parameter of the LSTM neural network is set to be inversely proportional to the QD value between the fusion feature annotation of the current input and the fusion feature annotation of the previous input; the QD value is the logarithm of the ratio of the values ​​of the two fusion feature annotations with the golden section constant as the base; the fusion feature annotation is a geometric feature matrix or a graph network feature.

8. The payment security monitoring method based on multimodal data according to claim 1, characterized in that: The determining, based on the health trend and the transaction data, a transaction security situation corresponding to the monitored object includes: Calculate the difference between the health condition parameter at a future time point in the health trend and the health reference parameter to obtain the health characterization parameter Inputting the transaction data into a trained transaction anomaly recognition neural network to obtain transaction anomaly recognition parameters; Determine whether the health characterization parameter is greater than a first parameter threshold, and obtain a first determination result; Determining whether the transaction anomaly identification parameter is greater than a second parameter threshold, and obtaining a second determination result; When both the first judgment result and the second judgment result are yes, it is determined that there is an abnormality in the transaction security situation corresponding to the monitored object.

9. A payment security monitoring system based on multimodal data, characterized in that: The system comprises: An acquisition module, used to obtain physiological data and transaction data of the monitored object; an extraction module, configured to extract multimodal features corresponding to the physiological data and the transaction data based on a geometric mean algorithm and a graph convolutional network; the multimodal features comprising a geometric mean feature matrix and graph network features; A prediction module, configured to predict the health trend of the monitored subject based on the multimodal features based on an LSTM neural network; The judgment module is used to judge the transaction security situation corresponding to the monitored object based on the health trend and the transaction data, and to issue an alarm or intervene when there is an abnormality in the transaction security situation.

10. A payment security monitoring system based on multimodal data, characterized in that: The system comprises: a memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the payment security monitoring method based on multimodal data as described in any one of claims 1-8.

Images