Method for real-time query and analysis of financial big data based on in-memory computing
By constructing dynamic memory indexes and memory index weights, the problem of memory computing latency caused by the expansion of fiscal data volume was solved, enabling rapid analysis and real-time response of fiscal data, and improving memory computing efficiency and data reliability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 焦作市亿众运输信息服务中心
- Filing Date
- 2026-04-08
- Publication Date
- 2026-07-03
AI Technical Summary
The massive increase in fiscal data volume leads to response delays or crashes in in-memory computing systems, and existing technologies struggle to achieve stability and efficiency in real-time analysis under conditions of limited memory resources.
By constructing dynamic memory indexes and memory index weights, and combining text analysis, semantic extraction, vector models, and clustering models, historical fiscal data is indexed and summarized, enabling accurate location and efficient backtracking of key fiscal events while avoiding memory overflow.
Under conditions of limited memory resources, it enables rapid analysis and real-time response of financial data, reduces the calculation of unnecessary data, and improves memory computing efficiency as well as data reliability and accuracy.
Smart Images
Figure CN122332384A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of in-memory computing, querying, and analysis technology for fiscal data, specifically a method for real-time querying and analysis of fiscal big data based on in-memory computing. Background Technology
[0002] A time-series database is a specialized database system that records data with timestamps as the primary key. Compared to the static data storage and management of traditional databases, time-series databases focus more on trend analysis of time evolution. Due to their high dependence on time, they generally need to be bound to high-precision timestamps at the millisecond or even microsecond level.
[0003] For fiscal data, in-memory computing can significantly improve the speed of early warning and backtracking of transaction data compared to traditional database computing. However, fiscal transaction data will continue to expand over time, while memory capacity is limited. Once real-time analysis involves fiscal data spanning multiple years, in-memory computing will inevitably lead to data overflow due to insufficient memory, which will cause system response delays or even crashes.
[0004] To address this, a real-time query and analysis method for fiscal big data based on in-memory computing is proposed. Summary of the Invention
[0005] The purpose of this invention is to provide a real-time query and analysis method for fiscal big data based on in-memory computing. By indexing and summarizing the analysis results of historical finances, the method aims to achieve rapid analysis of real-time data and reduce the retrieval of historical data.
[0006] To achieve the above objectives, the present invention provides the following technical solution:
[0007] Real-time query and analysis methods for fiscal big data based on in-memory computing include:
[0008] Acquire historical analysis data of historical financial data, wherein the historical analysis data includes at least the results of historical analysis data and the historical memory write volume;
[0009] A dynamic memory index is constructed based on the results of the historical analysis data, and an initial memory index weight is generated by combining the dynamic memory index and the historical financial data.
[0010] Optionally, the historical analysis results include at least fiscal data early warning information, fiscal data anomaly information, and fiscal data trend information;
[0011] The historical analysis results are semantically extracted using a text analysis model to generate time-series fiscal category information. This information is then vectorized using a semantic vector model. A clustering model is used to cluster the vectorized information, generating fiscal category clusters. A dynamic in-memory index of the historical analysis data is then constructed based on these clusters. This index generation from vectorized historical analysis results enables attribution analysis of the data, facilitating data summarization and organization. This allows for precise location and efficient backtracking of key fiscal events within limited memory resources. Furthermore, it ensures both real-time performance and stability during in-memory data computation, avoiding memory overflow caused by full loading.
[0012] Optionally, an initial memory index weight is generated by calculating the dynamic memory index and the historical financial data based on a memory weight function, wherein the expression for the memory weight function is:
[0013] ;
[0014] in, The initial memory index weight, The historical time decay function, Historical fiscal data, The cosine similarity function is used. For the i-th dynamic memory index, For the j-th dynamic memory index, This method employs an empirical adjustment function. It assigns differentiated time and relational weights to different time points based on the temporal importance of the time series data and the relevance of different analytical results to financial data. This enhances the reliability of historical data used in real-time calculations, further avoiding data overflow caused by full-data computation and preventing analysis results from interfering with the analysis.
[0015] The time series characteristics of the historical fiscal data are analyzed according to a statistical model. Based on the time series characteristics of the historical fiscal data and the dynamic memory index, the time series characteristics are classified by a time series analysis model. The initial memory index weight is dynamically adjusted according to the classification results to generate memory index weight.
[0016] Optionally, the time series features include at least periodic features, trend features, and random disturbance features;
[0017] The historical fiscal data is classified using a time series analysis model to obtain classification labels for the time series features; the classification labels are then vectorized using a vector model to generate classification vector labels.
[0018] The dynamic memory index is vectorized using a vector model to generate validation vector labels. The similarity between the validation vector labels and the classification vector labels is calculated using Euclidean distance. If the similarity is greater than or equal to a vector similarity threshold, the initial memory index weights are weighted accordingly; otherwise, the initial memory index weights are filtered out based on the similarity. Based on fiscal data sequences, and combined with a data labeling and classification mechanism, existing time-series data is automatically labeled and organized dynamically. This ensures that the index not only provides a basis for data summarization but also serves as a basis for automated data labeling. It guarantees that each index in the time-series database stores the most timely and analytically valuable fiscal data fragments, and also allows for time- and spatial contextualization customization for each piece of time-series fiscal data, further improving data reliability and accuracy.
[0019] Based on the historical memory write volume of the historical financial data, the historical financial data is windowed into blocks. The blocked historical financial data is repeatedly spliced together to generate memory length verification data. Based on the memory length verification data and the dynamic memory index, feature verification is performed through a time series analysis model. Based on the feature verification results, the windowing strategy is saved, windowing weights are generated, and the memory index weights are updated.
[0020] Optionally, a data volume block window is obtained based on the historical memory write volume, and the historical financial data is processed by window block processing based on the data volume block window to generate an equal-length window data sequence;
[0021] The historical memory write volume is used to generate sorted coordinates, which include time and write volume. Time and write volume coordinate axes are established, with time on the horizontal axis and write volume on the vertical axis. Multiple sets of change curves are obtained by fitting the sorted coordinates according to the SVM algorithm, and multiple change rates are obtained based on the multiple sets of change curves.
[0022] Using multiple rates of change as multiple data repetition expansion values for the equal-length window data sequence, the equal-length window data sequence is repeatedly expanded according to these multiple data repetition expansion values to generate multiple sets of memory length verification data. This block-based processing enables data size confirmation for data written to memory for computation, ensuring accurate allocation of memory resources without data overflow. Simultaneously, by dynamically adjusting the window size based on the rate of change, each data block achieves an optimal balance between temporal continuity and computational load, providing accurate data basis for in-memory computation.
[0023] Optionally, multiple sets of memory length verification data are classified according to the time series analysis model to obtain classification labels; the classification labels are vectorized using a vector model to generate classification vector labels.
[0024] The dynamic memory index is vectorized using a vector model to generate verification vector labels. Cosine similarity is used to calculate the vector category similarity between the verification vector labels and the classification vector labels for verification. The data duplication expansion value of the verification data with the highest vector category similarity is stored as the window segmentation strategy. After vectorizing the window segmentation strategy, the window segmentation weight is obtained based on the magnitude of the vectorized data duplication expansion value. The memory index weight is then updated by weighting the memory index weights according to the window segmentation weights. This method further verifies the rationality and dynamic adaptability of data segmentation, ensuring that the data segmented by window does not lose the accuracy of data analysis. Furthermore, using the index as verification labels saves on subsequent additional data processing costs and facilitates direct data summarization after analysis, thereby significantly improving the response speed and resource utilization of real-time financial data analysis.
[0025] Real-time financial data is mapped and analyzed based on the memory index weights.
[0026] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0027] 1. This application establishes an index based on the analysis results of historical financial data, which can summarize existing data and also facilitate the reduction of unnecessary data addition during real-time data in-memory computation, thereby avoiding memory overflow caused by the computation of large amounts of data. At the same time, summarizing data can also speed up the retrieval of data in in-memory computation, further improving the efficiency of in-memory computation. By using each index as a label to divide the data into blocks and confirm the repetition of blocks, it can ensure that the amount of data to be written to memory is minimized without destroying the original data's temporal sequence and depth feature integrity. In addition, using the index as verification data can further improve the reusability of data and reduce cost investment. It can also improve the reliability and usability of data based on the data characteristics of existing financial data in different scenarios. Attached Figure Description
[0028] Figure 1 This is a schematic diagram of the method flow of the present invention. Detailed Implementation
[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0030] This invention provides a method for real-time querying and analysis of fiscal big data based on in-memory computing, the technical solution of which is as follows:
[0031] A method for real-time query and analysis of fiscal big data based on in-memory computing, referenced Figure 1 As shown, it includes: acquiring historical analysis data of historical financial data, wherein the historical analysis data includes at least the historical analysis data results and historical memory write volume;
[0032] A dynamic memory index is constructed based on the results of the historical analysis data, and an initial memory index weight is generated by combining the dynamic memory index and the historical financial data.
[0033] Optionally, the historical analysis results include at least fiscal data early warning information, fiscal data anomaly information, and fiscal data trend information. Fiscal data anomaly information includes, for example, abnormal fluctuations in fiscal revenue and expenditure. Fiscal data trend information includes, for example, annual expenditure growth trends, cyclical changes in tax revenue, and evolution of the efficiency of special fund utilization. When semantically extracting the historical analysis results using a text analysis model, keywords are first extracted from the early warning information, anomaly information, and trend information. For example, from "a department's office expenses exceeded the budget by 15% in the third quarter," core semantic elements such as "office expenses," "exceeded budget," and "third quarter" are extracted. These are then mapped to a predefined fiscal category label system to generate time-series fiscal category information containing multi-dimensional attributes such as timestamp, event type, degree of impact, and related departments.
[0034] The historical analysis results are semantically extracted using text analysis models (including, but not limited to, GPT, RoBERTa, etc.) to generate time-series fiscal category information. This information is then vectorized using a semantic vector model (e.g., CLIP, etc.). Finally, the vectorized fiscal category information is clustered using clustering models (including, but not limited to, K-means, DBSCAN, etc.) to generate fiscal category clusters. Each fiscal category cluster is labeled with a corresponding type tag, such as "budget execution anomaly cluster," "special fund retention cluster," and "revenue forecast deviation cluster." A dynamic in-memory index of the historical analysis data is constructed based on these fiscal category clusters. This vectorization of historical analysis results to generate the index enables attribution analysis of the data, facilitating data summarization and organization. This allows for accurate location and efficient backtracking of key fiscal events within limited memory resources. It also ensures that real-time data computation in memory maintains both real-time performance and stability, avoiding memory overflow caused by full loading.
[0035] Optionally, an initial memory index weight is generated by calculating the dynamic memory index and the historical financial data based on a memory weight function, wherein the expression for the memory weight function is:
[0036] ;
[0037] in, The initial memory index weight, The historical time decay function, ,in The time interval between historical fiscal data and the current moment. The attenuation coefficient can be set differently based on the type of fiscal data and relevant expert experience. It is the natural logarithm. Historical fiscal data, The cosine similarity function is used. For the i-th dynamic memory index, For the j-th dynamic memory index, The adjustment function is set by experts based on their data experience. This method assigns differentiated time weights and relational weights to different time points based on the temporal importance of the time series itself and the relevance of different analytical results to financial data. This improves the reliability of historical data used when calculating real-time data and further avoids data overflow caused by full-data calculation and interference with the analysis results.
[0038] The time series characteristics of the historical fiscal data are analyzed according to a statistical model (e.g., STL model). Based on the time series characteristics of the historical fiscal data and the dynamic memory index, the time series characteristics are classified through a time series analysis model. The initial memory index weight is dynamically adjusted according to the classification results to generate memory index weight.
[0039] Optionally, the time series features include at least periodic features, trend features, and random disturbance features; periodic features reflect the seasonal patterns of fiscal revenue and expenditure, such as concentrated appropriations at the end of the quarter; trend features depict long-term growth or contraction trends, such as a slowdown in tax revenue growth for three consecutive years; and random disturbance features capture the impact of sudden events, such as a surge in temporary expenditures caused by the pandemic.
[0040] The historical fiscal data is classified using a time series analysis model (including, but not limited to, bidirectional LSTM, Transformer model, etc.) to obtain classification labels for the time series features. The model is a pre-trained model, trained by experts to annotate the data labels. The data labels can be simple numbers or data codes, and the labeled data is historical fiscal data. The classification labels are vectorized using a vector model (e.g., Gemini Embedding) to generate classification vector labels.
[0041] The dynamic memory index is vectorized using a vector model to generate validation vector labels. The distance similarity between the validation vector labels and the classification vector labels is calculated using Euclidean distance. If the distance similarity is greater than or equal to a vector similarity threshold, the initial memory index weights are weighted accordingly, specifically as follows:
[0042] ;
[0043] in, To strengthen or reduce the initial memory index weight , This represents the distance similarity.
[0044] If the distance similarity is less than the vector similarity threshold, the initial memory index weights are filtered out based on the distance similarity. The vector similarity threshold is set by an expert.
[0045] Based on fiscal data sequences, and combined with a data labeling and classification mechanism, existing time-series data is automatically labeled and organized. This allows the index to not only provide a basis for data summarization, but also a basis for automated data labeling. This ensures that each index in the time-series database stores the most timely and valuable fiscal data fragments. It also allows for time- and spatial contextualization customization for each piece of time-series fiscal data, further improving the reliability and accuracy of the data.
[0046] Based on the historical memory write volume of the historical financial data, the historical financial data is windowed into blocks. The blocked historical financial data is repeatedly spliced together to generate memory length verification data. Based on the memory length verification data and the dynamic memory index, feature verification is performed through a time series analysis model. Based on the feature verification results, the windowing strategy is saved, windowing weights are generated, and the memory index weights are updated.
[0047] Optionally, a data volume block window is obtained based on the historical memory write volume, and the historical financial data is processed by window block processing based on the data volume block window to generate an equal-length window data sequence;
[0048] The historical memory write volume is used to generate sorted coordinates, which include time and write volume. Time and write volume coordinate axes are established, with time on the horizontal axis and write volume on the vertical axis. Multiple sets of change curves are obtained by fitting the sorted coordinates according to the SVM algorithm, and multiple change rates are obtained based on the multiple sets of change curves.
[0049] Using multiple rates of change as multiple data repetition expansion values for the equal-length window data sequence, the equal-length window data sequence is repetitively expanded according to the multiple data repetition expansion values to generate multiple sets of memory length verification data; the specific expression is:
[0050] ;
[0051] in, Verify data for memory length. To augment values for repeated data, This is a data sequence with equal-length windows. By segmenting the data, the size of the data written to memory for computation can be confirmed, ensuring that memory resources are accurately allocated without overflowing. At the same time, the window size is dynamically adjusted based on the rate of change, so that each block of data achieves an optimal balance between temporal continuity and computational load, providing accurate data basis for in-memory computation.
[0052] Optionally, multiple sets of memory length verification data are classified according to the time series analysis model (including, but not limited to, bidirectional LSTM, Transformer model, etc.) to obtain classification labels; the classification labels are vectorized to generate classification vector labels through a vector model.
[0053] The dynamic memory index is vectorized using a vector model to generate verification vector labels. Cosine similarity is used to calculate the vector category similarity between the verification vector labels and the classification vector labels for verification. The data duplication expansion value of the verification data with the highest vector category similarity is stored as the window segmentation strategy. After vectorizing the window segmentation strategy, the window segmentation weight is obtained based on the magnitude of the vectorized data duplication expansion value. The memory index weight is then updated by weighting the memory index weights according to the window segmentation weights. This method further verifies the rationality and dynamic adaptability of data segmentation, ensuring that the data segmented by window does not lose the accuracy of data analysis. Furthermore, using the index as verification labels saves on subsequent additional data processing costs and facilitates direct data summarization after analysis, thereby significantly improving the response speed and resource utilization of real-time financial data analysis.
[0054] Real-time financial data is mapped and analyzed based on the memory index weights.
[0055] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A method for real-time querying and analysis of fiscal big data based on in-memory computing, characterized in that: include: Acquire historical analysis data of historical financial data, wherein the historical analysis data includes at least the results of historical analysis data and the historical memory write volume; A dynamic memory index is constructed based on the results of the historical analysis data, and an initial memory index weight is generated by combining the dynamic memory index and the historical financial data. The time series characteristics of the historical fiscal data are analyzed according to a statistical model. Based on the time series characteristics of the historical fiscal data and the dynamic memory index, the time series characteristics are classified by a time series analysis model. The initial memory index weight is dynamically adjusted according to the classification results to generate memory index weight. Based on the historical memory write volume of the historical financial data, the historical financial data is windowed into blocks. The blocked historical financial data is repeatedly spliced together to generate memory length verification data. Based on the memory length verification data and the dynamic memory index, feature verification is performed through a time series analysis model. Based on the feature verification results, the windowing strategy is saved, windowing weights are generated, and the memory index weights are updated. Real-time financial data is mapped and analyzed based on the memory index weights.
2. The method for real-time query and analysis of fiscal big data based on in-memory computing according to claim 1, characterized in that, A dynamic in-memory index is constructed based on the historical analysis data results. The initial in-memory index weights are generated by combining the dynamic in-memory index and the historical financial data, including: The historical analysis results include at least fiscal data early warning information, fiscal data anomaly information, and fiscal data trend information; The historical analysis results are semantically extracted using a text analysis model to generate time-series fiscal category information. This time-series fiscal category information is then vectorized using a semantic vector model. The vectorized time-series fiscal category information is then clustered using a clustering model to generate fiscal category clusters. Finally, a dynamic memory index of the historical analysis data is constructed based on these fiscal category clusters.
3. The method for real-time query and analysis of fiscal big data based on in-memory computing according to claim 1, characterized in that, The initial memory index weights are generated based on the dynamic memory index and the historical financial data using a memory weighting function. The expression for the memory weighting function is as follows: ； in, The initial memory index weight, The historical time decay function, Historical fiscal data, The cosine similarity function is used. For the i-th dynamic memory index, For the j-th dynamic memory index, This is an empirical adjustment function.
4. The method for real-time query and analysis of fiscal big data based on in-memory computing according to claim 1, characterized in that, The historical fiscal data is analyzed using a statistical model to determine its time-series characteristics. Based on these characteristics and the dynamic memory index, a time-series feature classification is performed using a time-series analysis model. The initial memory index weights are then dynamically adjusted based on the classification results to generate the memory index weights, including: The time series features include at least periodic features, trend features, and random disturbance features; The historical fiscal data is classified using a time series analysis model to obtain classification labels for the time series features; the classification labels are then vectorized using a vector model to generate classification vector labels. The dynamic memory index is vectorized using a vector model to generate validation vector labels. The distance similarity between the validation vector labels and the classification vector labels is calculated using Euclidean distance. If the distance similarity is greater than or equal to the vector similarity threshold, the initial memory index weights are weighted to enhance the weights based on the distance similarity. If the distance similarity is less than the vector similarity threshold, the initial memory index weights are filtered out based on the distance similarity.
5. The method for real-time query and analysis of fiscal big data based on in-memory computing according to claim 1, characterized in that, Based on the historical memory write volume of the historical financial data, the historical financial data is windowed into blocks. The block-processed historical financial data is then repeatedly concatenated to generate memory length verification data, including: Based on the historical memory write volume, a data volume block window is obtained, and based on the data volume block window, the historical financial data is processed into window blocks to generate an equal-length window data sequence. The historical memory write volume is used to generate sorted coordinates, which include time and write volume. Time and write volume coordinate axes are established, with time on the horizontal axis and write volume on the vertical axis. Multiple sets of change curves are obtained by fitting the sorted coordinates according to the SVM algorithm, and multiple change rates are obtained based on the multiple sets of change curves. Using multiple rates of change as multiple data repetition expansion values for the equal-length window data sequence, the equal-length window data sequence is repetitively expanded according to the multiple data repetition expansion values to generate multiple sets of memory length verification data.
6. The method for real-time query and analysis of fiscal big data based on in-memory computing according to claim 1, characterized in that, Based on the memory length verification data and the dynamic memory index, feature verification is performed using a time-series analysis model. The window segmentation strategy is saved based on the feature verification results, window segmentation weights are generated, and the memory index weights are updated, including: Based on the time-series analysis model, multiple sets of memory length verification data are classified to obtain classification labels; the classification labels are then vectorized using a vector model to generate classification vector labels. The dynamic memory index is vectorized using a vector model to generate verification vector labels. The similarity between the verification vector labels and the classification vector labels is calculated using cosine similarity for verification. The data duplication expansion value of the verification data with the largest vector category similarity is saved as the window segmentation strategy. After vectorizing the window segmentation strategy, the window segmentation weight is obtained based on the magnitude of the vectorized data duplication expansion value. The memory index weight is then updated by weighting the memory index weight based on the window segmentation weight.