Data analysis method and device, electronic equipment, storage medium and program product

Abstract

The embodiment of the invention provides a data analysis method and device, electronic equipment, a storage medium and a program product, and relates to the field of financial science and technology or other related fields. The method comprises the following steps: receiving a financial product set to be analyzed, a revenue function of the financial product set and a financial product sampling frequency; based on the financial product set and the number of financial product sampling times, generating a plurality of financial product subsets through a Monte Carlo sampling algorithm; dividing the plurality of financial product subsets to a parallel computing unit of a graphics processing unit (GPU); determining a Sharpley value of the financial product set based on the financial product subset and the revenue function through a parallel computing unit; and outputting an analysis result of the financial product set based on the Shapley value. According to the method, the accuracy of data analysis is improved, and the hardware resource consumption is reduced.

Description

Data analysis methods, devices, electronic equipment, storage media and software products Technical Field

[0001] This application relates to the field of financial technology or other related fields, and in particular to a data analysis method, apparatus, electronic device, storage medium and program product. Background Technology

[0002] The Shapley value is a mathematical model in cooperative game theory used to fairly distribute the returns of multiple participants in a cooperative scheme. The core principle of the Shapley value is to quantify the weight of each participant's contribution to the overall return by calculating the mean marginal contribution of each participant across all possible combinations of cooperation. In the fintech field, the Shapley value reflects the degree to which each financial product contributes to the overall return of a set of financial products. Therefore, the Shapley value can currently be used to characterize the results of data analysis on sets of financial products.

[0003] Currently, existing methods for calculating Shapley values ​​can be mainly divided into exact algorithms and approximate calculation methods based on Monte Carlo simulations.

[0004] However, calculating the Shapley value using precise algorithms typically involves high computational complexity, leading to significant hardware resource consumption. Existing estimation methods using approximate calculations suffer from poor accuracy. Therefore, using this Shapley value for data analysis of a set of financial products presents challenges such as high hardware resource consumption, low efficiency, and poor accuracy. Summary of the Invention

[0005] This application provides a data analysis method, apparatus, electronic device, storage medium, and program product to solve the technical problems of poor accuracy, low efficiency, and high hardware resource consumption in existing data analysis technologies.

[0006] In a first aspect, this application provides a data analysis method, the method comprising:

[0007] Receive the set of financial products to be analyzed, the return function of the set of financial products, and the number of times the financial products are sampled;

[0008] Based on the set of financial products and the number of times the financial products are sampled, multiple subsets of financial products are generated using the Monte Carlo sampling algorithm.

[0009] The subsets of financial products are divided into parallel computing units of the graphics processing unit (GPU).

[0010] The Sharpe ratio of the financial product set is determined by the parallel computing unit based on the subset of financial products and the return function.

[0011] Based on the Shapley value, the analysis results of the financial product set are output.

[0012] Secondly, this application provides a data analysis apparatus, the apparatus comprising:

[0013] The receiving module is used to receive the set of financial products to be analyzed, the return function of the set of financial products, and the number of times the financial products are sampled.

[0014] The processing module is configured to generate multiple subsets of financial products based on the set of financial products and the number of times the financial products are sampled, using a Monte Carlo sampling algorithm; divide the multiple subsets of financial products into parallel computing units of a graphics processing unit (GPU); and determine the Shapley value of the set of financial products based on the subsets of financial products and the return function using the parallel computing units.

[0015] The output module is used to output the analysis results of the financial product set based on the Shapley value.

[0016] Thirdly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0017] The memory stores computer-executed instructions;

[0018] The processor executes computer execution instructions stored in the memory to implement the method as described in any of the first aspects.

[0019] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any of the first aspects.

[0020] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method described in any one of the first aspects.

[0021] The data analysis method, apparatus, electronic device, storage medium, and program product provided in this application sample a set of financial products using the Monte Carlo algorithm by sampling the financial products a certain number of times, generating multiple subsets of financial products. This avoids the combination explosion problem of traversing all combinations in the exact algorithm, thus reducing computational complexity and hardware computing resource consumption. By dividing multiple subsets of financial products into GPU parallel computing units, the many-core architecture of the GPU is fully utilized to parallelize a large number of independent sampling computation tasks. The Shapley value is determined based on the payoff function by the parallel computing units, enabling fast calculation of the Shapley value, thereby improving the efficiency of outputting the analysis results of the financial product set based on the Shapley value. GPU parallelization provides more computational resources for the Monte Carlo-based Shapley value calculation, thus laying the foundation for increasing the number of Monte Carlo samplings, thereby improving the accuracy of the Shapley value calculation and the accuracy of the analysis results of the financial product set output based on the Shapley value. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0023] Figure 1 is a flowchart illustrating a data analysis method provided in this application;

[0024] Figure 2 is a schematic diagram of the architecture of a Monte Carlo Shapley value parallelization system based on GPU heterogeneous computing provided in this application;

[0025] Figure 3 is a schematic diagram of the structure of a data analysis device provided in this application;

[0026] Figure 4 is a schematic diagram of the hardware structure of an electronic device provided in this application.

[0027] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0028] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0029] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of the relevant data all comply with the relevant laws, regulations, and standards of the relevant countries and regions, have taken necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation access points for users to choose to authorize or refuse.

[0030] Furthermore, the technical solution involved in this application, which involves big data analysis of user information (including but not limited to personal biometrics, identity data, consumption data, asset data, electronic terminal operation data, etc.) and the use of artificial intelligence technology for automated decision-making, and makes decisions that have a significant impact on personal rights based on the results of automated decision-making, provides users with corresponding operation entry points for users to choose to agree to or reject the results of automated decision-making; if the user chooses to reject, the process will proceed to the expert decision-making process.

[0031] It should be noted that the data analysis methods, devices, electronic devices, storage media, and program products provided in this application can be used in the fintech field, or in any field other than fintech. The application fields of the data analysis methods, devices, electronic devices, storage media, and program products in this application are not limited.

[0032] The following is a brief explanation of some of the terms and concepts used in this application:

[0033] Shapley value: The Shapley value is a mathematical model in cooperative game theory used to fairly distribute the benefits of multi-participant cooperation. Its core principle is to quantify the weight of each participant's fair allocation of overall benefits by calculating the mean marginal contribution of each participant across all possible combinations of cooperation. This algorithm can be used to solve the problem of benefit distribution in multi-agent collaboration (such as distributed computing nodes and resource contributors). Unlike traditional average distribution, the Shapley value emphasizes the dynamic calculation of "contribution."

[0034] Monte Carlo simulation (Shapley scenario): Monte Carlo simulation is a numerical computation method based on random sampling and statistical analysis. In Shapley value calculation, directly traversing all combinations of participants (the combinatorial explosion problem) results in excessive computational cost. Therefore, the Monte Carlo method is used to randomly sample a subset for approximate calculation. Monte Carlo simulation is suitable for solving high-dimensional Shapley value problems (e.g., Shapley value calculations involving more than 10,000 participants).

[0035] In the fintech field, the Sharpe ratio can be used to reflect the contribution of each financial product in a set of financial products to the overall return of that set. Therefore, the Sharpe ratio can currently be used to characterize the results of data analysis on a set of financial products.

[0036] Currently, there are two main methods for calculating the Shapley value:

[0037] 1. Precise calculation of the Shapley value. In current cooperative game scenarios (such as distributed resource allocation and machine learning feature contribution analysis), the Shapley value is usually calculated using an exact algorithm, that is, by traversing all possible combinations of participants (…). Given a set of combinations (where N is the number of participants), calculate the marginal contribution of each participant in each subset, and finally take the mean as the Shapley value.

[0038] However, this method has high computational complexity. Specifically, the time complexity of this exact algorithm is O(log n). ( When the number of participants N exceeds 15, the computation time increases exponentially, failing to meet real-time requirements (such as profit and loss attribution under financial market risk). Hardware resource consumption is high: storing intermediate results for all combinations requires a large amount of memory; at N=20, more than 1 million data sets need to be stored, resulting in high hardware costs.

[0039] 2. Central Processing Unit (CPU) Serial Monte Carlo Simulation. For high-dimensional Shapley value calculation (e.g., N>20), existing techniques use Monte Carlo simulation for approximate calculation. By randomly sampling a subset of participants (e.g., 1% of the total number of combinations), the marginal contribution of each subset is calculated serially on the CPU, and finally the mean is calculated as the estimated value of Shapley.

[0040] However, this method suffers from a trade-off between accuracy and efficiency, as well as insufficient parallel computing capabilities. Specifically, the accuracy of Monte Carlo simulations depends on the number of samples, but CPU serial computation is inefficient; achieving 95% confidence requires several hours of computation, making it difficult to balance real-time performance and accuracy. Furthermore, the limited number of threads in CPU architectures (typically 4-64 threads) prevents full utilization of the parallelism of independent sampling tasks in Monte Carlo simulations, leading to idle computing resources.

[0041] Therefore, existing methods for calculating the Sharpe ratio suffer from high hardware resource consumption, low efficiency, and poor accuracy. Consequently, data analysis of financial product sets based on this Sharpe ratio also suffers from high hardware resource consumption, low efficiency, and poor accuracy.

[0042] Considering the aforementioned problems with existing data analysis methods, this application proposes a Monte Carlo Shapley value calculation method based on a parallelized architecture using a Graphics Processing Unit (GPU), and outputs analysis results for a set of financial products based on this Shapley value. GPU parallelization improves the efficiency of Shapley value calculation, thereby enhancing the efficiency of data analysis on the set of financial products based on this Shapley value. GPU parallelization also allows for greater computational resources in Monte Carlo Shapley value calculation, thus laying the foundation for increasing the number of Monte Carlo samplings and improving the accuracy of Shapley value calculation, ultimately enhancing the accuracy of data analysis on the set of financial products based on this Shapley value.

[0043] The data analysis methods, apparatus, electronic devices, storage media, and program products provided in this application are intended to solve the above-mentioned technical problems of the prior art.

[0044] Optionally, the execution entity of the data analysis method provided in this application can be any electronic device with processing capabilities, such as a terminal or a server. Alternatively, in some embodiments, the execution entity of the data analysis method can also be a data analysis system. This data analysis system can be deployed in a cloud environment or a server cluster, for example. This application does not limit the deployment environment of the data analysis system.

[0045] The following describes the technical solution of this application and how it solves the above-mentioned technical problems by taking an electronic device as the execution subject of the data analysis method provided in this application as an example, in conjunction with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0046] Figure 1 is a flowchart illustrating a data analysis method provided in this application. As shown in Figure 1, the method includes the following steps:

[0047] S101: Receive the set of financial products to be analyzed, the return function of the set of financial products, and the number of times the financial products are sampled.

[0048] For example, the aforementioned set of financial products can refer to the set of target objects to be analyzed, which may include any number of financial instruments, such as stocks, bonds, options, or combinations thereof. This set of financial products can be considered the "set of participants" in cooperative game theory.

[0049] Optionally, the above return function can be used as a mathematical function to quantify the returns of a collection of financial products.

[0050] Optionally, the sampling number of the aforementioned financial products can be the total sampling number T of the Monte Carlo simulation. This sampling number can be used to characterize the scale of the Monte Carlo simulation. For example, the larger the sampling number of the financial products, i.e., the larger the scale of the Monte Carlo simulation, the higher the accuracy of the Shapley value estimation result. The smaller the sampling number of the financial products, i.e., the smaller the scale of the Monte Carlo simulation, the lower the accuracy of the Shapley value estimation result.

[0051] Alternatively, the electronic device may receive, for example, the set of financial products to be analyzed, the return function of the set of financial products, and the number of times the financial products are sampled, through an application programming interface (API) or a graphical user interface (GUI).

[0052] S102. Based on the set of financial products and the number of times the financial products are sampled, multiple subsets of financial products are generated using the Monte Carlo sampling algorithm.

[0053] The Monte Carlo sampling algorithm approximates the Shapley value calculation by randomly generating multiple subsets of financial products from a set of financial products. These subsets can be used to characterize a possible combination of participants.

[0054] Optionally, the electronic device can utilize the parallel computing power of the GPU to generate multiple random subsets of financial products in parallel using a Monte Carlo sampling algorithm based on the set of financial products and the number of times the financial products are sampled. Alternatively, the electronic device can, for example, input the aforementioned set of financial products and the number of times the financial products are sampled into the Monte Carlo sampling algorithm to obtain multiple subsets of financial products.

[0055] Optionally, the specific implementation of the Monte Carlo sampling algorithm described above can refer to any existing Monte Carlo sampling algorithm process. For example, an electronic device can generate random numbers based on the Monte Carlo sampling algorithm, and generate multiple subsets of financial products based on these random numbers, the set of financial products, and the number of sampling times for the financial products. Optionally, each GPU thread can independently generate multiple random subsets of financial products, and the electronic device can convert the thread identifier (Identity document, ID) into a random number seed based on a hash function to ensure the randomness of the sampling.

[0056] S103. Divide multiple subsets of financial products into parallel computing units of the graphics processing unit (GPU).

[0057] Alternatively, the parallel computing unit can be, for example, a stream multitasking queue of the GPU. Or, the parallel computing unit can also be a Compute Unified Device Architecture (CUDA) thread block of the GPU.

[0058] Optionally, different parallel computing units can independently handle the computational tasks of one or more subsets of financial products. Electronic devices can partition multiple subsets of financial products across different parallel computing units of the GPU.

[0059] Alternatively, electronic devices can determine which parallel computing units a subset of financial products is assigned to, for example, by using load balancing implementation methods and the utilization rate of parallel computing units.

[0060] S104. Using parallel computing units, determine the Shapeli value of the financial product set based on a subset of financial products and a return function.

[0061] Optionally, for any parallel computing unit, the electronic device may first calculate the marginal contribution based on the aforementioned subset of financial products and the return function using that parallel computing unit. Then, the electronic device may obtain the Shapley value of the set of financial products based on this marginal contribution.

[0062] S105. Based on the Shapley value, output the analysis results of the financial product set.

[0063] Optionally, the electronic device may use the Shapley value as the analysis result of the aforementioned set of financial products. It should be understood that this application does not limit how the electronic device outputs the analysis result of the set of financial products. For example, the electronic device may display the analysis result of the set of financial products.

[0064] Alternatively, the electronic device can obtain the analysis results of the financial product set based on the aforementioned Shapley value by referring to any existing method of representing the analysis results of the financial product set based on the Shapley value, which will not be elaborated here.

[0065] In this embodiment, the Monte Carlo algorithm is used to sample the set of financial products by the number of sampling attempts, generating multiple subsets of financial products. This avoids the combination explosion problem of traversing all combinations in the exact algorithm, thus reducing computational complexity and hardware resource consumption. By dividing the multiple subsets of financial products into GPU parallel computing units, the many-core architecture of the GPU is fully utilized, parallelizing a large number of independent sampling computation tasks. The Shapley value is determined based on the payoff function by the parallel computing units, enabling fast calculation of the Shapley value and improving the efficiency of outputting the analysis results of the financial product set based on this Shapley value. GPU parallelization provides greater computational resource space for the Monte Carlo-based Shapley value calculation, thus laying the foundation for increasing the number of Monte Carlo sampling attempts and improving the accuracy of the Shapley value calculation. Therefore, the accuracy of the analysis results of the financial product set output based on this Shapley value is improved.

[0066] The following section details how electronic devices, through parallel computing units, determine the Shapley value of a set of financial products based on a subset of financial products and a return function:

[0067] As one possible implementation, for any financial product in the subset of financial products, the electronic device can, for example, use a parallel computing unit to calculate the marginal contribution of that financial product to the subset of financial products based on the aforementioned revenue function.

[0068] The marginal contribution value can be used to characterize the impact of the financial product on the returns of the aforementioned subset of financial products.

[0069] Optionally, the electronic device can calculate the marginal contribution value of the financial product to the subset of financial products based on the above-mentioned revenue function through a parallel computing unit. For example, it can refer to any existing method for calculating the marginal contribution value of a financial product to the set of financial products based on a revenue function, which will not be elaborated here.

[0070] Then, the electronic device can determine the Shapeli value of the financial product set based on the aforementioned marginal contribution value.

[0071] In some embodiments, the electronic device may write the aforementioned marginal contribution value into shared memory to further improve data analysis efficiency.

[0072] Then, the electronic device can asynchronously read the marginal contribution value in the shared memory through the CPU, and accumulate the marginal contribution value of each financial product in the financial product subset to the financial product subset, to obtain the marginal contribution accumulation result.

[0073] After obtaining the above-mentioned cumulative marginal contribution result, the electronic device can obtain the aforementioned Shapley value based on the cumulative marginal contribution result.

[0074] In this context, shared memory can be a memory space within the GPU that can be accessed by multiple parallel computing units. Optionally, the aforementioned shared memory can be, for example, the GPU's on-chip shared memory. Writing marginal contribution values ​​to shared memory can reduce global memory access latency and improve data read / write efficiency.

[0075] By asynchronously reading the marginal contribution value from shared memory, collaborative computing between the GPU and CPU can be achieved. Asynchronous reading means that after initiating a read operation, the CPU can continue performing other operations without waiting for the read operation to complete. This avoids the CPU being idle while waiting for data, thus improving the overall computational efficiency of the system.

[0076] By writing the marginal contribution value into shared memory using the above method, the frequent access latency of the GPU's global memory is reduced, enabling rapid data exchange and reuse within the GPU kernel, thereby improving the efficiency of the marginal contribution value aggregation process. The CPU asynchronously reads the marginal contribution values ​​from shared memory and accumulates them to obtain the cumulative marginal contribution result. This achieves asynchronous collaborative computing between the CPU and GPU, allowing the GPU to focus on highly parallel marginal contribution calculations, while the CPU handles subsequent accumulation and statistics. This avoids idle computing resources caused by the GPU waiting for data aggregation, fully utilizing the advantages of heterogeneous computing architecture and further improving the efficiency of Shapley value acquisition. Obtaining the Shapley value based on the cumulative marginal contribution result ensures the accuracy of the Shapley value calculation, thus laying the foundation for obtaining the analysis results of the financial product set.

[0077] In some embodiments, the electronic device may, for example, obtain the target marginal contribution accumulation result corresponding to the set of financial products based on the marginal contribution accumulation result corresponding to the multiple subsets of financial products.

[0078] Then, the electronic device can divide the sum of the target marginal contributions by the number of times the financial products are sampled (as mentioned above) to obtain the quotient as the aforementioned Sharpe ratio.

[0079] Using the above method, based on the cumulative marginal contributions of multiple subsets of financial products, the target cumulative marginal contribution of the entire set of financial products is obtained. This summarizes the intermediate results of all sampling batches, yielding the sum of the marginal contributions of all financial products. The Shapley value is obtained by dividing the target cumulative marginal contribution by the number of times the financial products are sampled, ensuring that the Shapley value meets the mathematical definition and guarantees its accuracy.

[0080] In this embodiment, for any financial product in the subset of financial products, the marginal contribution value of that financial product to the subset is calculated. This method accurately quantifies the contribution of a single financial product in a specific combination, ensuring the fairness and rationality of the final weight allocation. It reflects the principle of "allocation according to contribution" in cooperative game theory and improves the accuracy of determining the Shapley value of the set of financial products based on this marginal contribution value.

[0081] The following section details how electronic devices generate multiple subsets of financial products based on a set of financial products and the number of times these financial products are sampled, using the Monte Carlo sampling algorithm:

[0082] As one possible implementation, electronic devices can use the Monte Carlo sampling algorithm to call texture memory and sample the aforementioned set of financial products according to the number of times the financial products are sampled, thus obtaining multiple subsets of financial products.

[0083] Optionally, electronic devices can also store multiple subsets of financial products in CUDA Unified Memory.

[0084] For example, texture memory can be a dedicated, read-only cache memory on the GPU. Texture memory can provide efficient caching and data retrieval services.

[0085] For example, CUDA unified memory can be a memory management model provided by the CUDA programming model. CUDA unified memory can create a unified memory address space shared between the CPU and GPU, allowing processors in the system (such as the CPU or GPU) to access this data using the same pointers.

[0086] In this embodiment, the Monte Carlo sampling algorithm is used to access texture memory for sampling. The unique caching mechanism of texture memory optimizes the reading speed of data such as random numbers, reducing memory access latency. Multiple subsets of financial products are stored in CUDA unified memory, eliminating the explicit data copying overhead between the CPU and GPU, and achieving seamless data sharing and dynamic transfer between them. Through the combined effect of these memory optimization techniques, the bottleneck of data input and output is reduced, enabling the GPU's parallel computing units to acquire and process data more efficiently, thereby improving the overall efficiency of Monte Carlo sampling and Shapley value calculation.

[0087] As another possible implementation, before generating multiple subsets of financial products based on the set of financial products and the number of times the financial products are sampled using a Monte Carlo sampling algorithm, the electronic device may, for example, determine the batching of the number of times the financial products are sampled based on the occupancy of parallel computing units and / or the confidence level of the historical Shapley values ​​of the set of financial products.

[0088] Then, the electronic device can generate multiple subsets of financial products based on the set of financial products, the number of times the financial products are sampled, and the batches are segmented, using the Monte Carlo sampling algorithm.

[0089] In this context, "segmenting batches" can refer to dividing the total number of financial product samplings (e.g., 100,000) into smaller units or "batch processing" times. For example, 100,000 samplings can be divided into 10 batches, with each batch processing a subset of 10,000 financial products.

[0090] GPU utilization is used to characterize the ratio of the number of active warps in each stream processor to the maximum number of warps it can support. A higher utilization rate indicates that the GPU's computing resources are being utilized more fully. A lower utilization rate indicates that the GPU has more unused computing resources.

[0091] If the current batch size for sampling financial products results in low GPU utilization, it indicates that the workload per batch is too small to fully utilize the GPU's computing units. Therefore, reducing the batch size (i.e., increasing the number of samples processed per batch) can improve GPU utilization for individual batches.

[0092] The confidence level described above can be used to characterize the accuracy of historical Shapley values. When using the Monte Carlo sampling algorithm, the more samples taken, the higher the confidence level. The fewer samples taken, the lower the confidence level.

[0093] Taking the example of an electronic device determining the batch size of the financial product sampling based on the occupancy rate of parallel computing units and the confidence level of the historical Shapley values ​​of the financial product set before generating multiple subsets of financial products using the Monte Carlo sampling algorithm, the electronic device may first determine the initial batch size of the financial product sampling based on the occupancy rate of parallel computing units.

[0094] Then, the electronic device can increase the initial segmentation batch size when the confidence level of the historical Shapley value is less than the preset target confidence level.

[0095] The initial batch size is determined by the utilization rate of parallel computing units, ensuring optimized computational efficiency under current hardware resources. When the confidence level of historical Shapley values ​​is lower than the preset target confidence level, it indicates low confidence, requiring an increase in confidence. Therefore, the initial batch size can be increased to improve the accuracy of Shapley value calculation. Through this dynamic feedback mechanism, an adaptive match between computing resources and accuracy requirements is achieved, enhancing the flexibility of data analysis.

[0096] If the confidence level of the historical Shapley value is lower than the preset target confidence level, it indicates that the accuracy of the historical Shapley value is low, meaning that more sampling is needed. In this case, the initial batch size should be increased to improve the accuracy of the Shapley value.

[0097] By determining the batch size based on the utilization rate of parallel computing units and the confidence level of historical Shapley values, the system can dynamically and intelligently adjust the task granularity according to real-time computing load and accuracy requirements. For example, larger batches are used when GPU utilization is high to reduce scheduling overhead, while more batches are pre-planned to meet accuracy requirements when confidence is insufficient. Generating subsets of financial products based on these batch sizes ensures that computing tasks are rationally divided into scales suitable for GPU parallel processing, avoiding thread idleness or resource contention, thus guaranteeing the accuracy of the calculation process and improving the efficiency of Shapley value calculation.

[0098] Figure 2 is a schematic diagram of the architecture of a parallelized Monte Carlo Shapley value system based on GPU heterogeneous computing provided in this application. As shown in Figure 2, the system may include a task scheduling module, a parallelized sampling module, a marginal contribution calculation module, and a statistical aggregation module.

[0099] Task scheduling module: Receives a set of participants S (participants are, for example, financial products) and a profit function v (e.g., the profit function of a financial product), and dynamically allocates Monte Carlo sampling batches to GPU computing units.

[0100] Parallelized sampling module: Generates tens of thousands of random participant subsets on the GPU (e.g., batch processing 4096 sets each time), and achieves zero-copy memory access through CUDA kernel functions.

[0101] Marginal contribution calculation module: For each subset A⊆S, calculate the marginal contribution v(A∪{i})−v(A) of all participants i∈A in parallel, and use shared memory to optimize data reuse.

[0102] Statistical aggregation module: Accumulates sampling results from each batch in real time, asynchronously transmits data to the CPU through a double buffering strategy, and finally outputs a Shapley value vector.

[0103] Specifically, during initialization: input the participant set S, the revenue function v, and the total number of samplings T (e.g., T=10^6). Then, perform GPU task splitting: divide the T samplings into K batches (e.g., K=1000, 1000 samplings per batch), and allocate them to the GPU's Stream multitasking queue.

[0104] During parallel computing, for each batch, the following steps are performed: a. Randomly generate a subset A (which can be based on a pseudo-random number generator, with the seed obtained by hashing the GPU thread ID); b. Calculate the marginal contribution for all i∈S (using GPU warp-level parallel computing); c. Write the result (i.e. the marginal contribution) to the global memory buffer.

[0105] Then, the results are aggregated, meaning the CPU asynchronously reads data from the buffer and updates the Shapley value accumulator until all batches are completed. Using the above method, the accumulated result can be divided by T to obtain the final Shapley value estimate vector, which is then output.

[0106] In terms of memory optimization, CUDA unified memory can be used to avoid explicit data transfer between the CPU and GPU, and texture memory can be used to accelerate random number reading. In addition, dynamic load balancing can be used, which automatically adjusts the batch size based on GPU utilization (e.g., SM utilization > 90%) to prevent thread bundle splitting.

[0107] In terms of accuracy control, for example, confidence intervals can be calculated in real time, and if the target accuracy (such as 95%) is not achieved, additional sampling batches (key CUDA kernel functions) can be added automatically.

[0108] In this embodiment, the inventors' actual experiments showed that GPU parallelization increases Monte Carlo sampling speed by 50-100 times (in actual tests with N=20, 10^6 sampling times were reduced from 3.2 hours on CPU to 2.1 minutes on GPU). Regarding resource utilization optimization, by decoupling memory bandwidth from computing units, GPU utilization reaches over 85% (compared to <30% for CPU serial utilization). In terms of accuracy controllability, it supports dynamically adding sampling batches, increasing confidence from 80% to 90% within the same timeframe (e.g., 5 minutes) compared to existing technologies. Regarding scalability, it supports distributed multi-GPU expansion, linearly increasing computational scale (in actual tests, 4 GPUs in parallel can achieve real-time computation in a 40-participant system).

[0109] Figure 3 is a schematic diagram of the structure of a data analysis device provided in this application. As shown in Figure 3, the data analysis device 30 may include: a receiving module 31, a processing module 32, and an output module 33.

[0110] The receiving module 31 is used to receive the set of financial products to be analyzed, the return function of the set of financial products, and the number of sampling times of the financial products.

[0111] Processing module 32 is used to generate multiple subsets of financial products based on the set of financial products and the number of sampling times of the financial products using the Monte Carlo sampling algorithm; to divide the multiple subsets of financial products into the parallel computing units of the graphics processing unit (GPU); and to determine the Shapley value of the set of financial products based on the subsets of financial products and the return function using the parallel computing units.

[0112] Output module 33 is used to output the analysis results of the financial product set based on the Sharpe ratio.

[0113] Optionally, the processing module 32 is specifically used to calculate the marginal contribution value of any financial product in the subset of financial products to the subset of financial products through a parallel computing unit based on the return function; the marginal contribution value is used to characterize the impact of the financial product on the return of the subset of financial products; and the Shapley value of the set of financial products is determined based on the marginal contribution value.

[0114] Optionally, the processing module 32 is specifically used to write the marginal contribution value into the shared memory; asynchronously read the marginal contribution value in the shared memory through the central processing unit (CPU), and accumulate the marginal contribution value of each financial product in the financial product subset to obtain the marginal contribution accumulation result; and obtain the Shapley value based on the marginal contribution accumulation result.

[0115] Optionally, the processing module 32 is specifically used to obtain the target marginal contribution accumulation result corresponding to the financial product set based on the marginal contribution accumulation result corresponding to multiple financial product subsets; and to divide the target marginal contribution accumulation result by the quotient obtained by the number of samplings of financial products as the Shapley value.

[0116] Optionally, processing module 32 is specifically used to use the Monte Carlo sampling algorithm to call the texture memory and sample the set of financial products according to the number of times the financial products are sampled, to obtain multiple subsets of financial products. Optionally, processing module 32 is also used to store the multiple subsets of financial products into the CUDA unified memory of the computing unified device architecture.

[0117] Optionally, processing module 32 is further configured to determine the batching of financial product sampling times based on the occupancy rate of parallel computing units and / or the confidence level of the historical Shapley values ​​of the financial product set before generating multiple subsets of financial products using a Monte Carlo sampling algorithm based on the financial product set and the number of financial product sampling times. Optionally, processing module 32 is specifically configured to generate multiple subsets of financial products using a Monte Carlo sampling algorithm based on the financial product set, the number of financial product sampling times, and the batching.

[0118] Optionally, the processing module 32 is specifically used to determine the initial segmentation batch of the sampling number of financial products based on the occupancy rate of the parallel computing unit; when the confidence level of the historical Shapley value is less than the preset target confidence level, the initial segmentation batch is increased to the segmentation batch.

[0119] Optionally, the parallel computing unit is a GPU's Stream multitasking queue.

[0120] The data analysis device 30 provided in this application is used to execute the aforementioned data analysis method embodiments. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0121] Figure 4 is a schematic diagram of the hardware structure of an electronic device provided in this application. The electronic device 40 shown in Figure 4 includes a memory 41, a processor 42, and a communication interface 43. The memory 41, processor 42, and communication interface 43 are communicatively connected to each other. For example, the memory 41, processor 42, and communication interface 43 can be connected via a network. Alternatively, the electronic device 40 may also include a bus 44. The memory 41, processor 42, and communication interface 43 are communicatively connected to each other via the bus 44. Figure 4 shows an electronic device 40 in which the memory 41, processor 42, and communication interface 43 are communicatively connected to each other via a bus 44.

[0122] The memory 41 can be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 41 can store programs, and when the program stored in the memory 41 is executed by the processor 42, the processor 42 and the communication interface 43 are used to execute the data analysis method of any of the foregoing embodiments. The memory can also store data required by the data analysis method.

[0123] The processor 42 can be a general-purpose CPU, microprocessor, application-specific integrated circuit (ASIC), graphics processing unit (GPU), or one or more integrated circuits.

[0124] Processor 42 can also be an integrated circuit chip with signal processing capabilities. In implementation, the data analysis method of this application can be completed through the integrated logic circuits in the hardware of processor 42 or through software instructions. The aforementioned processor 42 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the following embodiments of this application. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the following embodiments of this application can be directly implemented by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory 41, and processor 42 reads the information in memory 41 and, in conjunction with its hardware, completes the data analysis method of this application.

[0125] Communication interface 43 uses transceiver modules, such as, but not limited to, transceivers, to enable communication between electronic device 40 and other devices or communication networks. For example, data sets can be acquired through communication interface 43.

[0126] When the aforementioned electronic device 40 includes a bus 44, the bus 44 may include a path for transmitting information between various components of the electronic device 40 (e.g., memory 41, processor 42, communication interface 43).

[0127] This application also provides a computer-readable storage medium, which may include various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk. Specifically, the computer-readable storage medium stores program instructions, which are used in the methods described in the above embodiments.

[0128] This application also provides a program product including executable instructions stored in a readable storage medium. At least one processor of an electronic device can read the executable instructions from the readable storage medium, and the at least one processor executes the executable instructions to cause the electronic device to implement the data analysis methods provided in the various embodiments described above.

[0129] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0130] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0131] The term "multiple" in this document refers to two or more. The term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Furthermore, the character " / " in this document generally indicates an "or" relationship between the preceding and following related objects; in formulas, " / " indicates a "division" relationship. Additionally, it should be understood that in the description of this application, words such as "first" and "second" are used only for descriptive purposes and should not be construed as indicating or implying relative importance or order.

[0132] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application.

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A data analysis method, characterized in that, The method includes: receiving a set of financial products to be analyzed, a return function of the set of financial products, and the number of times the financial products are sampled; generating multiple subsets of financial products based on the set of financial products and the number of times the financial products are sampled using a Monte Carlo sampling algorithm; allocating the multiple subsets of financial products to parallel computing units of a graphics processing unit (GPU); determining the Shapley value of the set of financial products based on the subsets of financial products and the return function using the parallel computing units; and outputting the analysis results of the set of financial products based on the Shapley value.

2. The method according to claim 1, characterized in that, The step of determining the Shapley value of the financial product set based on the subset of financial products and the return function through the parallel computing unit includes: for any financial product in the subset of financial products, calculating the marginal contribution value of the financial product to the subset of financial products through the parallel computing unit based on the return function; the marginal contribution value is used to characterize the impact of the financial product on the return of the subset of financial products; and determining the Shapley value of the financial product set based on the marginal contribution value.

3. The method according to claim 2, characterized in that, The step of determining the Shapley value of the financial product set based on the marginal contribution value includes: writing the marginal contribution value into shared memory; asynchronously reading the marginal contribution value in the shared memory through a central processing unit (CPU), and accumulating the marginal contribution value of each financial product in the financial product subset to the financial product subset to obtain a marginal contribution accumulation result; and obtaining the Shapley value based on the marginal contribution accumulation result.

4. The method according to claim 3, characterized in that, The step of obtaining the Shapley value based on the accumulated marginal contribution result includes: obtaining the target accumulated marginal contribution result corresponding to the set of financial products based on the accumulated marginal contribution result corresponding to the subset of financial products; and dividing the target accumulated marginal contribution result by the number of times the financial products are sampled, and using the quotient as the Shapley value.

5. The method according to any one of claims 1-4, characterized in that, The method of generating multiple subsets of financial products based on the set of financial products and the number of samplings of the financial products using a Monte Carlo sampling algorithm includes: using the Monte Carlo sampling algorithm to call texture memory and sample the set of financial products according to the number of samplings of the financial products to obtain the multiple subsets of financial products; the method further includes: storing the multiple subsets of financial products in the CUDA unified memory of the computing unified device architecture.

6. The method according to any one of claims 1-4, characterized in that, Before generating multiple subsets of financial products based on the set of financial products and the number of times the financial products are sampled using the Monte Carlo sampling algorithm, the method further includes: determining the batching of the number of times the financial products are sampled based on the utilization rate of the parallel computing unit and / or the confidence level of the historical Shapley value of the set of financial products; generating multiple subsets of financial products based on the set of financial products and the number of times the financial products are sampled using the Monte Carlo sampling algorithm includes: generating the multiple subsets of financial products based on the set of financial products, the number of times the financial products are sampled, and the batching using the Monte Carlo sampling algorithm.

7. The method according to claim 6, characterized in that, Based on the utilization rate of the parallel computing unit and the confidence level of the historical Shapley value of the financial product set, the batch of sampling times for the financial products is determined, including: determining the initial batch of sampling times for the financial products based on the utilization rate of the parallel computing unit; and increasing the initial batch to the batch size when the confidence level of the historical Shapley value is less than the preset target confidence level.

8. The method according to any one of claims 1-4, characterized in that, The parallel computing unit is the GPU's Stream multitasking queue.

9. A data analysis device, characterized in that, The apparatus includes: a receiving module for receiving a set of financial products to be analyzed, a return function of the set of financial products, and the number of sampling times for the financial products; a processing module for generating multiple subsets of financial products based on the set of financial products and the number of sampling times for the financial products using a Monte Carlo sampling algorithm; dividing the multiple subsets of financial products into parallel computing units of a graphics processing unit (GPU); determining the Shapley value of the set of financial products based on the subsets of financial products and the return function using the parallel computing units; and an output module for outputting the analysis results of the set of financial products based on the Shapley value.

10. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 8.

12. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 8.

Images