Financial data analysis engine based on big data

Abstract

The invention discloses a financial data analysis engine based on big data, which extracts a first financial record and a second financial record to be analyzed from a financial database and accurately matches the first financial record and the second financial record. For completely consistent records, the system prompts an administrator to confirm whether to delete redundant entries so as to ensure the cleanliness of a data set; for records which are not completely consistent but possibly have semantic association, deep learning and natural language processing technologies are further introduced to carry out semantic coding on the first financial record and the second financial record so as to extract fine-grained semantic information in the first financial record and the second financial record; and a feature alignment mode is adopted to capture the nuisance between the two, and then semantic repetition recognition is performed on the nuisance to confirm whether one of the two needs to be deleted or not. In this way, the relation between the financial records can be more accurately understood, and more reliable data support is provided for decision making.

Description

Technical Field

[0001] The present application relates to the field of data analysis, and more specifically, to a financial data analysis engine based on big data. Background Art

[0002] In today's wave of digital transformation, the scale of enterprise operations continues to expand, and the complexity of business is increasing, resulting in an explosive growth in the amount of financial data generated. Due to the diverse sources and formats of financial data, as well as the heterogeneity between systems, there are often a large number of redundant, repeated, or semantically similar but not completely consistent records in financial databases, which seriously hinders the accuracy and analysis efficiency of financial data. Therefore, in order to maintain the accuracy and cleanliness of financial data, it is particularly important to deduplicate and clean financial records. With the development of this trend, traditional financial data analysis methods have gradually revealed their limitations. Traditional methods mainly rely on manual review and basic data mining tools, such as structured query language (SQL) and other traditional data processing methods. Although these methods can extract information from the database and conduct preliminary comparison and analysis, it is difficult to ensure that all relevant financial records are accurately identified when faced with a large number of scattered data sources.

[0003] For example, when it comes to massive data sources, traditional methods cannot effectively deal with the matching difficulties caused by differences in the format of financial records generated by different departments or different time periods. This may result in certain key financial entries not being correctly matched, which in turn affects the final analysis results. In addition, even for those records that can be successfully matched, traditional systems can usually only provide surface-level comparisons, that is, checking whether the two transactions are the same in basic elements such as amount and date, but cannot go deep into a more granular level for semantic understanding. This shallow matching method means that two records that appear to be consistent but actually have subtle but important differences - such as slightly different words in the description, or different business logic behind them - may be ignored. These subtle differences may reflect different economic activities or business contexts and are crucial for financial analysis and decision-making.

[0004] Therefore, we look forward to a financial data analysis engine based on big data. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a financial data analysis engine based on big data, which extracts the first and second financial records to be analyzed from the financial database and accurately matches the two. For completely consistent records, the system will prompt the administrator to confirm whether to delete redundant entries to ensure the cleanliness of the data set; for records that are not completely consistent but may have semantic associations, deep learning and natural language processing technologies are further introduced to semantically encode the first and second financial records respectively to extract the fine-grained semantic information in the first and second financial records respectively, and use feature alignment to capture the subtle differences between the two, and then confirm whether one of them needs to be deleted by semantically identifying the subtle differences. In this way, the relationship between financial records can be understood more accurately, providing more reliable data support for decision-making.

[0006] According to one aspect of the present application, a financial data analysis engine based on big data is provided, which includes: A financial record extraction module, used to extract a first financial record and a second financial record from a financial database; A financial record matching module, used for accurately matching the first financial record and the second financial record to obtain an accurate matching result; a duplicate record management module, configured to, in response to the accurate matching result being that the first financial record and the second financial record are completely identical, pop up the first financial record and the second financial record and ask the administrator to confirm whether to delete one of them; a semantic encoding module, configured to, in response to the precise matching result indicating that the first financial record and the second financial record are not completely consistent, semantically encode the first financial record and the second financial record to obtain a first financial record semantic encoding feature and a second financial record semantic encoding feature; A feature alignment module, configured to perform feature alignment based on semantic information guidance on the first financial record semantic coding feature and the second financial record semantic coding feature to obtain fine-grained alignment coding features between financial records; The matching result generating module is used to confirm whether to delete one of the first financial record and the second financial record based on the fine-grained alignment coding features between the financial records.

[0007] Compared with the prior art, the present application provides a financial data analysis engine based on big data, which extracts the first and second financial records to be analyzed from the financial database and accurately matches the two. For completely identical records, the system will prompt the administrator to confirm whether to delete redundant entries to ensure the cleanliness of the data set; for records that are not completely identical but may have semantic associations, deep learning and natural language processing technologies are further introduced to semantically encode the first and second financial records respectively to extract the fine-grained semantic information in the first and second financial records respectively, and use feature alignment to capture the subtle differences between the two. Then, the semantic duplication of the subtle differences is identified to confirm whether one of them needs to be deleted. In this way, the relationship between financial records can be understood more accurately, providing more reliable data support for decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0009] Figure 1 is a block diagram of a financial data analysis engine based on big data according to an embodiment of the present application; Figure 2 is a block diagram of a feature alignment module in a financial data analysis engine based on big data according to an embodiment of the present application; Figure 3 It is a block diagram of a fine-grained semantic information field construction unit of semantic features of financial records in a big data-based financial data analysis engine according to an embodiment of the present application. DETAILED DESCRIPTION

[0010] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0011] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0012] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0013] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0014] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0015] In the technical solution of the present application, a financial data analysis engine based on big data is proposed. Figure 1 FIG. 1 is a block diagram of a financial data analysis engine based on big data according to an embodiment of the present application. Figure 1 As shown, according to an embodiment of the present application, a financial data analysis engine 300 based on big data includes: a financial record extraction module 310, which is used to extract a first financial record and a second financial record from a financial database; a financial record matching module 320, which is used to accurately match the first financial record and the second financial record to obtain an accurate matching result; a duplicate record management module 330, which is used to pop up the first financial record and the second financial record in response to the accurate matching result that the first financial record and the second financial record are completely consistent, and the administrator confirms whether to delete one of them; a semantic coding module 340, which is used to semantically encode the first financial record and the second financial record in response to the accurate matching result that the first financial record and the second financial record are not completely consistent, to obtain a first financial record semantic coding feature and a second financial record semantic coding feature; a feature alignment module 350, which is used to perform feature alignment based on semantic information guidance on the first financial record semantic coding feature and the second financial record semantic coding feature to obtain a fine-grained alignment coding feature between financial records; a matching result generation module 360, which is used to confirm whether to delete one of the first financial record and the second financial record based on the fine-grained alignment coding feature between financial records.

[0016] In particular, the financial record extraction module 310 is used to extract the first financial record and the second financial record from the financial database. Among them, the financial record contains information such as transaction date, transaction amount, transaction description, account number, etc., and is a written record or electronic document of economic matters such as income, expenditure, assets, liabilities, etc. generated by the company in its daily business activities. They are important bases for the company's management to understand the company's financial status, evaluate performance, and make strategic decisions. They are also key information for external stakeholders such as investors, creditors, tax authorities, etc. to review the financial health of the company. Therefore, ensuring the integrity and accuracy of financial records is crucial to the normal operation of the company. Therefore, in the technical solution of the present application, first, the first financial record and the second financial record are extracted from the financial database.

[0017] In particular, the financial record matching module 320 is used to accurately match the first financial record and the second financial record to obtain an accurate matching result. It should be understood that in the actual operation of analyzing financial data, financial records may have problems such as repeated entry, incorrect input, inconsistent formats, etc. If these problems are not solved promptly and effectively, they will directly affect the authenticity and reliability of financial statements. For example, a sales transaction may appear twice in two different systems due to different entry habits of sales personnel, or the name of the same supplier may be slightly different in the records of different departments, resulting in multiple appearances of the same entity in the financial books. These situations not only increase the risk of redundant data, but may also mislead enterprises to make wrong decisions. In the technical solution of the present application, in order to avoid the occurrence of the above problems, the first financial record and the second financial record are accurately matched to automatically identify financial records from different sources but essentially the same. Through accurate matching, it can be ensured that each analyzed object is unique and representative, thereby improving the credibility and practicality of the overall analysis results. In an embodiment of the present application, the accurate matching is a de-duplication algorithm based on a complete match of field values.

[0018] In one example, the key to accurate matching lies in detailed comparison at the field level. For each financial record to be compared, the system will check the key information it contains one by one, such as transaction date, transaction amount, account number, transaction description, etc. The selection of these fields is based on their importance in distinguishing different financial activities. For example, transaction amount and date are usually regarded as hard constraints, because if two transactions are exactly the same in these two aspects, they are likely to represent the same economic event. The account number helps to confirm the specific parties involved in the transaction, further narrowing the scope of matching. As for the transaction description, although there may be some flexibility, it also provides important clues, especially when capturing records that appear to be the same on the surface but are actually different. Through such field-level comparisons, the system can effectively screen out those records that are highly similar or even completely consistent.

[0019] In particular, the duplicate record management module 330 is used to pop up the first financial record and the second financial record in response to the accurate matching result that the first financial record and the second financial record are completely consistent, and the administrator confirms whether to delete one of them. It should be understood that completely consistent records will have a negative impact on the financial management of the enterprise. For example, on the one hand, redundant data increases storage costs and processing time; on the other hand, more importantly, these duplicate records will affect the authenticity of financial statements and mislead management to make wrong decisions. For example, when calculating the total amount of income or expenditure, if all records are included in the statistics without discrimination, the final result will be twice or more than the actual situation, which poses a serious threat to key activities such as the company's budget planning and performance evaluation. Therefore, for financial records that are identified as completely consistent after accurate matching, the system can automatically pop up to remind the administrator, and the administrator can confirm whether to keep one of the records or delete all of them according to the actual situation. In this way, the efficiency of the automated process is guaranteed without losing flexibility, and the accuracy of the final result is ensured.

[0020] In particular, the semantic coding module 340 is used to semantically encode the first financial record and the second financial record to obtain the semantic coding feature of the first financial record and the semantic coding feature of the second financial record in response to the precise matching result that the first financial record and the second financial record are not completely consistent. If the precise matching result is that the first financial record and the second financial record are not completely consistent, the first financial record and the second financial record are semantically encoded to obtain the semantic coding feature of the first financial record and the semantic coding feature of the second financial record. It should be understood that when two financial records are judged to be "not completely consistent", it means that although they may be the same or very close in some basic elements (such as amount, date, etc.), there are significant differences in other aspects. For example, the text in the transaction description is slightly different, or the business logic behind it is different. These subtle differences may reflect different economic activities or business backgrounds, which are crucial for financial analysis and decision-making. In the technical solution of the present application, first, the first financial record and the second financial record are semantically encoded to extract the deep semantic meaning of the first financial record and the second financial record, which not only helps to more accurately identify those financial records that are similar on the surface but actually have subtle differences, but also provides a solid foundation for subsequent more complex analysis. In a specific example of the present application, the first financial record and the second financial record are semantically encoded by using a financial record semantic encoder based on the BERT (Bidirectional Encoder Representations from Transformers) model to obtain the semantic encoding features of the first financial record and the semantic encoding features of the second financial record. BERT can not only recognize simple vocabulary matching, but also understand the logical relationship within and between sentences. In this way, it can help the system distinguish between records that appear similar on the surface but actually represent different economic activities. In an example, "a payment to supplier A" and "a payment received from supplier A", although both involve "supplier A" and "a payment", the business behaviors behind them are obviously very different.

[0021] In particular, the feature alignment module 350 is used to perform feature alignment based on semantic information guidance on the semantic coding features of the first financial record and the semantic coding features of the second financial record to obtain fine-grained alignment coding features between financial records. Since financial records from different sources may contain multiple different types of information (such as amount, date, description, etc.), simply comparing their semantic coding as a whole may lead to misjudgment. Therefore, a more sophisticated comparison method is needed to examine the similarity between the two financial records dimension by dimension. Since the first financial record and the second financial record may have different data structures and formats, direct position-by-position semantic interaction may produce information differences, affecting the accuracy of feature comparison analysis. Based on this, in the technical solution of the present application, the semantic coding features of the first financial record and the semantic coding features of the second financial record are aligned based on semantic information guidance to obtain fine-grained alignment coding features between financial records. By constructing and utilizing the semantic information field, the subtle semantic relationship between the first financial record and the second financial record can be captured at a better level, thereby generating a highly comprehensive and semantically rich feature representation. In this way, it can be ensured that the financial records can be analyzed at a deeper level, thereby improving the accuracy of the final recognition. In a specific example of the present application, such as Figure 2 As shown, the feature alignment module 350 includes: a financial record semantic feature fine-grained semantic information field construction unit 351, which is used to construct a first financial-second financial record semantic feature fine-grained semantic information field between the first financial record semantic coding vector and the second financial record semantic coding vector; a financial record feature alignment unit 352, which is used to perform feature alignment on the first financial record semantic coding vector and the second financial record semantic coding vector based on the first financial-second financial record semantic feature fine-grained semantic information field to obtain an aligned first financial record semantic coding vector and an aligned second financial record semantic coding vector; a feature fine-grained response interaction unit 353, which is used to perform feature fine-grained response interaction on the aligned first financial record semantic coding vector and the aligned second financial record semantic coding vector to obtain a fine-grained alignment coding vector between financial records as the fine-grained alignment coding feature between financial records.

[0022] Specifically, the financial record semantic feature fine-grained semantic information field construction unit 351 is used to construct the first financial-second financial record semantic feature fine-grained semantic information field between the first financial record semantic encoding vector and the second financial record semantic encoding vector. More specifically, in a specific example of the present application, Figure 3As shown, the financial record semantic feature fine-grained semantic information field construction unit 351 includes: a feature dimension modulation subunit 3511, used to perform feature dimension modulation on the first financial record semantic coding vector and the second financial record semantic coding vector to obtain a modulated first financial record semantic coding vector and a modulated second financial record semantic coding vector, wherein the modulated first financial record semantic coding vector and the modulated second financial record semantic coding vector have the same feature dimension; a semantic interaction coding subunit 3512, used to perform convolution kernel attention-based semantic interaction coding on the modulated first financial record semantic coding vector and the modulated second financial record semantic coding vector to obtain the first financial-second financial record semantic feature fine-grained semantic information field.

[0023] More specifically, the feature dimension modulation subunit 3511 is used to perform feature dimension modulation on the first financial record semantic coding vector and the second financial record semantic coding vector to obtain the modulated first financial record semantic coding vector and the modulated second financial record semantic coding vector, wherein the modulated first financial record semantic coding vector and the modulated second financial record semantic coding vector have the same feature dimension. That is, the first financial record semantic coding vector and the second financial record semantic coding vector are subjected to feature dimension modulation based on point convolution to obtain the modulated first financial record semantic coding vector and the modulated second financial record semantic coding vector. It should be understood that in the process of aligning the first financial record semantic coding vector and the second financial record semantic coding vector between features, it is first necessary to ensure that the input feature vectors have the same feature dimension. Therefore, first, the first financial record semantic coding vector and the second financial record semantic coding vector are subjected to feature dimension modulation based on point convolution so that the first financial record semantic coding vector and the second financial record semantic coding vector have the same feature dimension, and the features on each dimension are more focused on specific semantic information. For example, in financial records, some dimensions may focus more on numerical information such as amount and date, while other dimensions focus on key words or business logic in transaction descriptions. In a specific example, the first financial record semantic coding vector and the second financial record semantic coding vector are modulated by the following feature dimension modulation formula to obtain the modulated first financial record semantic coding vector and the modulated second financial record semantic coding vector; wherein the feature dimension modulation formula is:

[0024]

[0025] in, represents the semantic encoding vector of the first financial record, represents the semantic encoding vector of the second financial record, represents point convolutional coding, is the activation function, represents the modulated semantic coding vector of the first financial record, Represents the modulated second financial record semantic coding vector.

[0026] More specifically, the semantic interaction encoding subunit 3512 is used to perform semantic interaction encoding based on convolution kernel attention on the modulated first financial record semantic encoding vector and the modulated second financial record semantic encoding vector to obtain the first financial-second financial record semantic feature fine-grained semantic information field. In an embodiment of the present application, first, the modulated first financial record semantic encoding vector and the modulated second financial record semantic encoding vector are fine-grained associated encoding to obtain the first financial-second financial record semantic feature fine-grained association matrix. That is, after completing the feature vector dimension adjustment, the modulated first financial record semantic encoding vector and the modulated second financial record semantic encoding vector are fine-grained associated encoding to capture the subtle semantic relationship between the modulated first financial record semantic encoding vector and the modulated second financial record semantic encoding vector to obtain the first financial-second financial record semantic feature fine-grained association matrix. Then, the first financial-second financial record semantic feature fine-grained association matrix is ​​input into the semantic information field encoding network based on the convolution kernel to obtain the first financial-second financial record semantic feature fine-grained semantic information field. Here, a semantic information field encoding network based on convolution kernels is used to capture semantic association information of different scales and levels in the first financial-second financial record semantic feature fine-grained association matrix. Further, by stacking multiple convolution layers, the system can gradually learn the semantic offset information between the first financial record and the second financial record, and realize the mapping of low-level features to high-level semantic concepts to obtain the first financial-second financial record semantic feature fine-grained semantic information field. In a specific example, the semantic interaction encoding formula is used to perform semantic interaction encoding based on convolution kernel attention on the modulated first financial record semantic encoding vector and the modulated second financial record semantic encoding vector to obtain the first financial-second financial record semantic feature fine-grained semantic information field; wherein, the semantic interaction encoding formula is:

[0027] in, yes The transposed vector of is matrix multiplication, yes The characteristic scale value of is a convolutional code with a convolution kernel of 3×3. represents the first financial-second financial record semantic feature fine-grained association matrix, A fine-grained semantic information field representing the semantic features of the first financial-second financial records.

[0028] Specifically, the financial record feature alignment unit 352 is used to perform feature alignment on the first financial record semantic coding vector and the second financial record semantic coding vector based on the first financial-second financial record semantic feature fine-grained semantic information field to obtain the aligned first financial record semantic coding vector and the aligned second financial record semantic coding vector. In this process, the modulated first financial record semantic coding vector and the modulated second financial record semantic coding vector can be respectively mapped to the inter-feature fine-grained semantic information field. In this process, each position feature in each financial record semantic coding vector will be adjusted according to the semantic strength and correlation of the corresponding position in the semantic information field so that the financial record semantic coding vector better matches the semantic distribution in the semantic information field, thereby achieving semantic alignment between the first financial record and the second financial record, and obtaining the aligned first financial record semantic coding vector and the aligned second financial record semantic coding vector. In a specific example, based on the first financial-second financial record semantic feature fine-grained semantic information field, the first financial record semantic encoding vector and the second financial record semantic encoding vector are feature aligned using the following feature alignment formula to obtain the aligned first financial record semantic encoding vector and the aligned second financial record semantic encoding vector; wherein the feature alignment formula is:

[0029]

[0030] in, represents the semantic encoding vector of the first financial record after alignment, The aligned second financial record semantic encoding vector is represented.

[0031] Specifically, the feature fine-grained response interaction unit 353 is used to perform feature fine-grained response interaction on the aligned first financial record semantic coding vector and the aligned second financial record semantic coding vector to obtain a fine-grained alignment coding vector between financial records as the fine-grained alignment coding feature between financial records. In an embodiment of the present application, the position-weighted sum between the aligned first financial record semantic coding vector and the aligned second financial record semantic coding vector is calculated to obtain the fine-grained alignment coding vector between financial records. In this way, the generated fine-grained alignment coding vector between financial records not only retains the semantic information of the original financial record, but also further quantifies the similarity between the two financial records, thereby enhancing the ability to recognize subtle differences between different financial records. In a specific example, the following feature fine-grained response interaction formula is used to perform feature fine-grained response interaction on the aligned first financial record semantic coding vector and the aligned second financial record semantic coding vector to obtain a fine-grained alignment coding vector between financial records; wherein, the fine-grained response interaction formula is:

[0032] in, and is the weight factor, Represents the fine-grained alignment encoding vector between the financial records.

[0033] In particular, the matching result generation module 360 ​​is used to confirm whether to delete one of the first financial record and the second financial record based on the fine-grained alignment coding features between the financial records. In an embodiment of the present application, first, the fine-grained alignment coding vector between the financial records is input into a classifier-based financial record semantic duplication identifier to obtain a recognition result, and the recognition result is used to indicate whether the first financial record and the second financial record are semantically consistent. That is, the fine-grained alignment coding features between the financial records obtained by matching the semantic information response using the first financial record semantic coding vector and the second financial record semantic coding vector are classified and processed, so as to intelligently generate a recognition result indicating whether the first financial record and the second financial record are semantically consistent. Semantic duplication recognition is performed by matching the features of the financial records at a fine granularity, thereby improving the accuracy of recognition. Further, in response to the recognition result that the first financial record and the second financial record are semantically consistent, the first financial record and the second financial record are popped up and the administrator confirms whether to delete one of them.

[0034] In a specific example of the present application, the labels of the classifier include whether the first financial record and the second financial record are semantically consistent (first label), and whether the first financial record and the second financial record are semantically inconsistent (second label), wherein the classifier determines which classification label the fine-grained alignment encoding vector between the financial records belongs to by a soft maximum function. It is worth noting that the first label p1 and the second label p2 here do not contain artificially set concepts. In fact, during the training process, the computer model does not have the concept of "whether the first financial record and the second financial record are semantically consistent". It only has two classification labels and the probability of the output feature under these two classification labels, that is, the sum of p1 and p2 is one. Therefore, the classification result of whether the first financial record and the second financial record are semantically consistent is actually converted into a binary classification class probability distribution that conforms to natural laws through the classification label. In essence, the physical meaning of the natural probability distribution of the label is used, rather than the linguistic text meaning of "whether the first financial record and the second financial record are semantically consistent".

[0035] Preferably, inputting the fine-grained alignment encoding vector between the financial records into a classifier-based financial record semantic repetition identifier to obtain an identification result comprises: The distance between each pair of eigenvalues ​​of the fine-grained alignment encoding vector between the financial records, such as the L2 distance, is calculated, and the square root of the distance is taken to obtain the distance representation matrix of the fine-grained alignment encoding between the financial records, that is,

[0036] in, represents the fine-grained alignment encoding vector between the financial records, and denote the i-th and j-th eigenvalues ​​of the fine-grained alignment encoding vector between the financial records, respectively. represents the distance between the i-th and j-th eigenvalues ​​of the fine-grained alignment encoding vectors between the financial records, represents the fine-grained alignment encoding distance representation matrix between the financial records, The fine-grained alignment encoding distance between the financial records represents the first The element value at position; The financial record fine-grained alignment coding autocorrelation matrix of the financial record fine-grained alignment coding vector as a row vector is obtained, that is, ; The fine-grained alignment coding vector between financial records is matrix-multiplied with the fine-grained alignment coding distance representation matrix between financial records to obtain the first-level mapping vector of the fine-grained alignment coding between financial records, that is, ; The first-level mapping vector of the fine-grained alignment coding between financial records is matrix-multiplied with the matrix product of the distance representation matrix of the fine-grained alignment coding between financial records and the self-correlation matrix of the fine-grained alignment coding between financial records to obtain a multi-level mapping vector of the fine-grained alignment coding between financial records. ; The multi-level mapping vector of the fine-grained alignment code between financial records is interpolated with the associated eigenvector of the fine-grained alignment code between financial records composed of the eigenvalues ​​of the auto-association matrix of the fine-grained alignment code between financial records to obtain an optimized fine-grained alignment code vector between financial records, wherein interpolation or zero padding is performed when the eigenvalue is insufficient; and The optimized fine-grained alignment encoding vector between financial records is input into a classifier-based financial record semantic repetition identifier to obtain an identification result.

[0037] Considering that the first financial record semantic coding vector and the second financial record semantic coding vector respectively represent the text semantic coding features of the first financial record and the second financial record, when performing the feature alignment interaction module based on the semantic information field, the difference in expression form between the first financial record and the second financial record will make the fine-grained alignment of the semantic information field blurred, resulting in the lack of fine-grained interaction feature instance determination of the fine-grained alignment coding vectors between the financial records, thereby affecting the accuracy of the recognition results obtained by the classifier-based financial record semantic repetition identifier.

[0038] Therefore, through the linear target mapping representation based on the similarity distance representation matrix of the fine-grained aligned coding vectors between the financial records, the complete similarity instantiation of the self-correlation of the fine-grained aligned coding vectors between the financial records is instantiated by a secondary target mapping representation based on a multi-level distribution hierarchy, and the negative impact factor of the association mismatch is compensated by the association fusion kernel bias to improve the degree of instance determination of the eigenvalues ​​of the fine-grained aligned coding vectors between the financial records under similarity constraints, that is, the degree of significance of the eigenvalues ​​as instances for classification and regression decisions, thereby improving the accuracy of the recognition results of the fine-grained aligned coding vectors between the financial records obtained by the classifier-based financial record semantic repetition identifier.

[0039] As described above, the financial data analysis engine 300 based on big data according to the embodiment of the present application can be implemented in various wireless terminals, such as a server with a financial data analysis algorithm based on big data. In a possible implementation, the financial data analysis engine 300 based on big data according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the financial data analysis engine 300 based on big data can be a software module in the operating system of the wireless terminal, or can be an application developed for the wireless terminal; of course, the financial data analysis engine 300 based on big data can also be one of the many hardware modules of the wireless terminal.

[0040] Alternatively, in another example, the big data-based financial data analysis engine 300 and the wireless terminal may also be separate devices, and the big data-based financial data analysis engine 300 may be connected to the wireless terminal via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0041] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A financial data analysis engine based on big data, characterized in that: include: A financial record extraction module, used to extract a first financial record and a second financial record from a financial database; A financial record matching module, used for accurately matching the first financial record and the second financial record to obtain an accurate matching result; a duplicate record management module, configured to, in response to the accurate matching result being that the first financial record and the second financial record are completely identical, pop up the first financial record and the second financial record and ask the administrator to confirm whether to delete one of them; a semantic encoding module, configured to, in response to the precise matching result indicating that the first financial record and the second financial record are not completely consistent, semantically encode the first financial record and the second financial record to obtain a first financial record semantic encoding feature and a second financial record semantic encoding feature; A feature alignment module, configured to perform feature alignment based on semantic information guidance on the first financial record semantic coding feature and the second financial record semantic coding feature to obtain fine-grained alignment coding features between financial records; The matching result generating module is used to confirm whether to delete one of the first financial record and the second financial record based on the fine-grained alignment coding features between the financial records.

2. The financial data analysis engine based on big data according to claim 1, characterized in that: The semantic encoding module is used to: Inputting the first financial record into a financial record semantic encoder based on a Bert model to obtain a first financial record semantic encoding vector as the first financial record semantic encoding feature; The second financial record is input into the financial record semantic encoder based on the Bert model to obtain a second financial record semantic encoding vector as the second financial record semantic encoding feature.

3. The financial data analysis engine based on big data according to claim 2, characterized in that: The feature alignment module comprises: A financial record semantic feature fine-grained semantic information field construction unit, used to construct a first financial-second financial record semantic feature fine-grained semantic information field between the first financial record semantic encoding vector and the second financial record semantic encoding vector; A feature alignment unit, configured to perform feature alignment on the first financial record semantic encoding vector and the second financial record semantic encoding vector based on the first financial-second financial record semantic feature fine-grained semantic information field to obtain an aligned first financial record semantic encoding vector and an aligned second financial record semantic encoding vector; A feature fine-grained response interaction unit is used to perform feature fine-grained response interaction on the aligned first financial record semantic coding vector and the aligned second financial record semantic coding vector to obtain a fine-grained alignment coding vector between financial records as the fine-grained alignment coding feature between financial records.

4. The financial data analysis engine based on big data according to claim 3, characterized in that: The financial record semantic feature fine-grained semantic information field construction unit includes: a feature dimension modulation subunit, configured to perform feature dimension modulation on the first financial record semantic coding vector and the second financial record semantic coding vector to obtain a modulated first financial record semantic coding vector and a modulated second financial record semantic coding vector, wherein the modulated first financial record semantic coding vector and the modulated second financial record semantic coding vector have the same feature dimension; The semantic interaction encoding subunit is used to perform semantic interaction encoding based on convolution kernel attention on the modulated first financial record semantic encoding vector and the modulated second financial record semantic encoding vector to obtain a fine-grained semantic information field of the first financial-second financial record semantic features.

5. The financial data analysis engine based on big data according to claim 4, characterized in that: The characteristic dimension modulation subunit is used to: The first financial record semantic coding vector and the second financial record semantic coding vector are modulated based on feature dimensions of point convolution to obtain the modulated first financial record semantic coding vector and the modulated second financial record semantic coding vector.

6. The financial data analysis engine based on big data according to claim 5, characterized in that: The semantic interaction encoding subunit is used to: Performing fine-grained association coding on the modulated first financial record semantic coding vector and the modulated second financial record semantic coding vector to obtain a first financial-second financial record semantic feature fine-grained association matrix; The first financial-second financial record semantic feature fine-grained association matrix is ​​input into a convolution kernel-based semantic information field encoding network to obtain the first financial-second financial record semantic feature fine-grained semantic information field.

7. The financial data analysis engine based on big data according to claim 6, characterized in that: The feature fine-grained response interaction unit is used to: A position-weighted sum between the aligned first financial record semantic encoding vector and the aligned second financial record semantic encoding vector is calculated to obtain a fine-grained alignment encoding vector between the financial records.

8. The financial data analysis engine based on big data according to claim 7, characterized in that: The matching result generating module comprises: A financial record semantic duplication identification unit, used for inputting the fine-grained alignment encoding vector between the financial records into a classifier-based financial record semantic duplication identifier to obtain an identification result, wherein the identification result is used to indicate whether the first financial record and the second financial record are semantically consistent; The duplicate record deletion confirmation unit is used to pop up the first financial record and the second financial record in response to the recognition result that the first financial record and the second financial record are semantically consistent, and allow the administrator to confirm whether to delete one of them.

9. The financial data analysis engine based on big data according to claim 8, characterized in that: The financial record semantic duplication identification unit is used to: Using multiple fully connected layers of the classifier to fully connect the fine-grained alignment encoding vector between the financial records to obtain an encoding classification feature vector; The encoded classification feature vector is passed through the Softmax classification function of the classifier to obtain the recognition result.

Images