Business and financial fusion method and system based on deep learning

Abstract

The invention discloses a business and financial fusion method and system based on deep learning, and the method comprises the following steps: S1, obtaining business and financial data, and carrying out the preprocessing; s2, coding business and financial data by adopting a double-tower structure, and establishing a semantic mapping relationship through cross attention alignment operation; s3, inputting the matching feature matrix into a Transform-XL network, and carrying out context relation modeling by introducing a gating memory mechanism and periodic position coding; s4, inputting the fusion semantic representation into the LoRA-BERT network, and executing fine granularity alignment and heterogeneous feature fusion; s5, constructing a supervised training sample, and jointly optimizing network parameters; and S6, performing integrated deployment on the trained dual networks to realize a whole business and financial fusion process. According to the invention, intelligent alignment and automatic reconciliation of heterogeneous financial data are realized, the reconciliation accuracy and processing efficiency are improved, and the manual intervention cost is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of enterprise digital management, and particularly relates to an industry and finance integration method and system based on deep learning. BACKGROUND

[0002] In the current enterprise digital transformation process, industry and finance integration as the core link connecting business operation and financial accounting has become a key means to improve management efficiency, strengthen business analysis and optimize resource allocation. Traditional industry and finance integration relies on rule-driven data connection, manually set field mapping and process docking method, which is difficult to meet the actual needs of increasingly complex business forms, highly heterogeneous data structures and dynamic evolution of financial system. In practical application, the data generated by the business processes of the enterprise's warehouse in and out, material management, budget execution, cost accounting, etc. are distributed in multiple systems, and there are obvious differences in data structure, semantic dimension and time granularity, which leads to great challenges in accurate mapping and real-time reconciliation between business data and financial data.

[0003] In the prior art, the mainstream industry and finance integration scheme generally adopts a data integration method based on ETL tools to extract and clean form data in the business system and then push it to the financial platform. This method has large workload in the initial construction stage and high maintenance cost, and cannot adapt to dynamically changing data models and processes. In addition, some systems attempt to achieve automatic reconciliation through rule engine-based semantic mapping, but in actual deployment, the rule coverage is limited, the context semantic understanding ability is weak, and it is difficult to identify the semantic association of the deep business logic, and the processing capacity for abnormal reconciliation and complex multi-business scenarios is insufficient.

[0004] With the development of artificial intelligence technology, especially the wide application of deep learning in natural language processing, graph structure modeling and semantic representation, a new technical path is provided for industry and finance integration. Some research attempts to introduce neural networks for business and financial data matching modeling, such as vectorizing business text through the Transformer model, or using graph neural networks to capture the structural dependency relationship between business nodes. However, these methods still have many shortcomings when applied to real enterprise scenarios. On the one hand, most existing models deal with homogeneous structure data, and have weak feature alignment capability between multiple heterogeneous data sources such as warehouse in and out systems, smart procurement platforms and financial sharing platforms, making it difficult to accurately establish high-quality semantic mapping between business actions and financial vouchers. On the other hand, the models generally rely on large-scale training samples, while the industry and finance reconciliation data available for annotation within the enterprise is limited, resulting in unstable model training quality and poor generalization ability.

[0005] In addition, although traditional deep learning models such as standard BERT or Transformer have strong representation ability, they have memory decay problem when facing long sequence context, and it is difficult to completely model the cross-period budget flow, inventory record or multi-round financial adjustment behavior. In terms of structure design, the standard BERT model has large number of parameters and high deployment cost, and is not suitable for efficient integration and real-time calling in the existing system architecture of enterprises. At the same time, most of the existing methods cannot effectively process the multi-granularity semantic alignment demand, and the field information such as materials, uses and operators involved in business semantics has obvious hierarchical structure. If not finely disassembled, semantic confusion may occur in the reconciliation stage.

[0006] Therefore, how to provide a deep learning-based industry and finance integration method and system, which can integrate long sequence modeling capability, fine-grained alignment mechanism and low parameter adaptation structure, realize heterogeneous alignment and intelligent reconciliation of business semantics and financial semantics, is a problem that those skilled in the art need to solve. SUMMARY

[0007] One object of the present application is to provide a deep learning-based industry and finance integration method and system. The present application makes full use of the long dependence modeling capability of the Transformer-XL network and the low rank adaptation mechanism of the LoRA-BERT network, and describes in detail the whole process from preprocessing, semantic matching, context modeling to fine-grained alignment and fusion of business data and financial data, which has the advantages of high semantic understanding accuracy, strong heterogeneous data reconciliation capability and low deployment cost.

[0008] According to the deep learning-based industry and finance integration method of the embodiment of the present application, the following steps are included:

[0009] S1, obtaining industry and finance data and preprocessing;

[0010] S2, encoding the preprocessed industry and finance data using a double-tower structure, and establishing a semantic mapping relationship between business features and financial features through cross-attention alignment operation, and outputting a matching feature matrix;

[0011] S3, inputting the matching feature matrix into the Transformer-XL network, modeling the context relationship by introducing the gating memory mechanism and the cycle position encoding, and generating a fusion semantic representation with global dependence;

[0012] S4, inputting the fusion semantic representation into the lightweight low-rank adaptation LoRA-BERT network, performing fine-grained alignment and heterogeneous feature fusion in the multi-granularity semantic space, and outputting the industry and finance reconciliation score result and the reconciliation label;

[0013] S5, construct a supervised training sample, and take the industry and financial reconciliation score results and reconciliation labels as a supervision signal, jointly optimize the gating memory parameters in the Transformer-XL network and the low-rank mapping parameters in the LoRA-BERT network, and complete the dual-network joint training;

[0014] S6, integrate the trained dual network for deployment, perform parallel coding, context modeling and reconciliation prediction on the newly added industry and financial data, and realize the whole process of industry and financial integration.

[0015] Optionally, the industry and financial data includes business data and financial data, the business data includes warehouse in-out records, material use information, and inventory change logs, and the financial data includes voucher records, budget execution data, and cost accounting details.

[0016] Optionally, the preprocessing includes field mapping, timestamp alignment, missing value filling and data standardization.

[0017] Optionally, the S2 specifically includes:

[0018] S21, respectively embedding coding the preprocessed business data and financial data, constructing a business feature matrix B=[b1, b2,…,bm] and a financial feature matrix F=[f1, f2,…,fn], wherein bi represents the i-th business feature vector, fj represents the j-th financial feature vector, m represents the number of business features, and n represents the number of financial features. m n i j

[0019] S22, inputting the business feature matrix B and the financial feature matrix F into a cross-attention alignment network, establishing a semantic mapping relationship between the business features and the financial features, calculating a semantic correlation matrix M between each pair of business-financial features, and constructing an attention matching function as follows:

[0020]

[0021] Wherein, M(i,j) represents an element value in the semantic correlation matrix, Q b (i) represents a query vector mapped by the i-th business feature vector b i , K f (j) represents a key vector mapped by the j-th financial feature vector f j , d k represents the vector dimension, γ represents the distance penalty adjustment coefficient, represents the Euclidean distance square between the i-th business feature vector and the j-th financial feature vector, for quantifying the nonlinear semantic difference;

[0022] ​​​​S23, based on the semantic correlation matrix M, calculate the matching feature matrix Z, and construct the following feature mapping function:

[0023]

[0024] wherein z i represents the i-th matching feature vector, h i represents the i-th service feature vector after fusion, alpha ij is the attention weight normalized by the Softmax function, W h ,W r respectively represent different projection mapping matrices, b z represents the matching bias vector, and sigma represents a nonlinear activation function.

[0025] Optionally, the S3 specifically comprises:

[0026] S31, input the matching feature matrix Z into the Transformer-XL network, and introduce a gating memory mechanism and a residual connection at each layer to construct a context representation:

[0027]

[0028] wherein, represents the i-th matching feature vector at the l-th layer, represents the i-th matching feature vector at the corresponding layer, represents the inherited gating memory state from the historical position, u represents the gating control vector, W1, W2, W u respectively represent different weight matrices, b1, b u is a bias vector, sigma represents a nonlinear activation function, represents a corresponding element multiplication operation, tanh(·) represents a hyperbolic tangent function;

[0029] S32, add the context representation to the periodic position encoding vector p i to construct a fused semantic representation s i , and the calculation method is:

[0030]

[0031] wherein s i represents the i-th fused semantic representation, sin(·) represents a hyperbolic sine function, omega represents a periodic encoding frequency factor, psi represents a periodic offset term, i represents a position number, and rho represents a normalization function.

[0032] Optionally, the fused semantic representation is constructed by introducing a gating memory mechanism to construct a multi-layer residual recurrent structure, performing position-aware context modeling on the matching feature matrix, performing weighted fusion based on the current state and historical memory state in each layer, introducing a periodic position encoding vector, constructing a periodic dynamic position function using a learnable frequency factor and offset term to enhance time-aware capability, maintaining semantic continuity and cross-step dependency structure consistency in cross-layer output, and finally generating a fused semantic representation sequence.

[0033] Optionally, the S4 specifically includes:

[0034] S41, inputting the fused semantic representation s i into a lightweight low-rank adaptive LoRA-BERT network, performing low-rank attention transformation on each s i to construct the following embedding expression: i i-w i i+w , and performing low-rank attention transformation on each s i .

[0035]

[0036] wherein v i represents the i-th LoRA-transformed context-enhanced feature representation, a ik is the attention weight normalized by M(i, k) through the Softmax function, A u represents the LoRA up-mapping matrix, A d represents the LoRA down-mapping matrix, w represents the window width, and LayerNorm(·) represents the layer normalization operation.

[0037] S42, inputting each v i into a multi-granularity feature fusion structure within a BERT model to perform three-layer fine-grained alignment operations at the sentence level, sub-structure level and field level, perform heterogeneous projection on each semantic position, and construct the following heterogeneous alignment expression:

[0038] e i = σ(W3·v i +W4·f λ(i) +W5·z i +b e ).

[0039] λ(i) = (i x p) mod n.

[0040] wherein e i represents the i-th heterogeneous alignment output vector, λ(·) represents the index mapping function, f λ(i) ​​​represents the i-th financial feature vector, p represents the alignment step size, z i represents the i-th matching feature vector, W3, W4, W5 represent different fusion weight matrices respectively, b e represents the fusion bias vector, σ represents the nonlinear activation function, mod is the modulus operator, i represents the position number, and n represents the number of financial features.

[0041] S43, output vectors e i are integrated to generate the industry and financial account reconciliation score vector and the reconciliation label, respectively, and the output function is defined as follows:

[0042]

[0043] where r i represents the i-th reconciliation score result, t i represents the i-th reconciliation classification label, which takes the value of 1 or 2, W s represents the score projection matrix, b s is the score bias vector, W c represents the classification weight matrix, b c is the classification bias, σ represents the nonlinear activation function, and argmax is the argument value when the function takes the maximum value.

[0044] Optionally, the LoRA-BERT network is constructed based on a pre-trained BERT model, a low-rank parameter pair composed of a trainable dimension-increasing mapping matrix and a dimension-reducing mapping matrix is introduced in each layer of attention calculation, the fused semantic representation is taken as input, the self-attention encoding operation within the local window is performed on the premise that the BERT model backbone parameters remain unchanged, and the financial feature vector and the matching feature vector are integrated through a residual connection structure to output a fine-grained heterogeneous fusion result for account reconciliation scoring and label generation.

[0045] Optionally, the S5 specifically includes:

[0046] S51, based on the industry and financial account reconciliation score vector and the reconciliation label, a supervised training sample set D= where q represents the total number of training samples.

[0047] S52, a joint loss function is constructed Optimize the gating memory parameter set Θ in the Transformer-XL network G and the low-rank mapping parameter set Θ in the LoRA-BERT network L , and the joint loss expression is defined as follows:

[0048]

[0049] wherein, log2(·) represents a logarithm function, λ represents a score confidence adjustment coefficient, u ti represents an account reconciliation label t i corresponding label vector, V represents a score conversion matrix, b v represents a score bias vector, μ represents a cross-model regularization penalty coefficient, A G represents a gated attention residual mapping matrix in the Transformer-XL network, A L represents a low-rank fusion projection matrix in the LoRA-BERT network, represents a memory representation generated by the i-th matching feature representation through the Transformer-XL network, represents an alignment representation generated by the i-th matching feature representation through the LoRA-BERT network, represents the square of the Euclidean norm of a vector, used to quantify the difference;

[0050] S53, using a joint back propagation mechanism, based on the gradient descent optimization strategy of minimizing the loss function jointly update the parameter set Θ G and Θ L , using a gradient descent method based on the learning rate η to iteratively update, complete the synchronous training of the double network.

[0051] According to an embodiment of the present application, a deep learning-based industry and finance integration system comprises:

[0052] A data processing module is configured to acquire industry and finance data and perform preprocessing.

[0053] An encoding mapping module is configured to encode the preprocessed industry and finance data using a double-tower structure, and establish a semantic mapping relationship between business features and financial features through cross-attention alignment operations, and output matching feature representations.

[0054] A context modeling module is configured to input the matching feature representations into a Transformer-XL network, and perform context relationship modeling through the introduction of a gated memory mechanism and a periodic position encoding, to generate fusion semantic representations with global dependencies.

[0055] An alignment fusion module is configured to input the fusion semantic representations into a lightweight low-rank adaptive LoRA-BERT network, perform fine-grained alignment and heterogeneous feature fusion in a multi-granularity semantic space, and output industry and finance account reconciliation score results and account reconciliation labels.

[0056] The supervision training module is used for constructing a supervision training sample, taking the industry and financial account reconciliation score result and the reconciliation label as a supervision signal, and jointly optimizing the gated memory parameter in the Transform-XL network and the low-rank mapping parameter in the LoRA-BERT network to complete the double-network joint training.

[0057] The deployment application module is used for integrating and deploying the double network after the training, performing parallel coding, context modeling and reconciliation prediction on the newly added industry and financial data, and realizing the whole process of industry and financial integration.

[0058] The beneficial effects of the present application are:

[0059] Firstly, the present application realizes high-precision semantic alignment between business features and financial features by constructing a double-tower structure and introducing a cross-attention mechanism, effectively solves the problem that multi-source heterogeneous data is difficult to match in the prior art, and significantly improves the automatic association efficiency between the in-out warehouse behavior and the financial voucher.

[0060] Secondly, the present application introduces the Transform-XL network with long-distance dependence modeling capability to model the context of the industry and financial semantic sequence, and combines the period position coding and the gated memory mechanism, so that the model can accurately capture the cross-period semantic relationship in budget flow, inventory evolution and financial change, thereby improving the stability and continuity of the reconciliation modeling.

[0061] Finally, the present application integrates the LoRA low-rank adaptation mechanism in the BERT model structure, combines the fine-grained semantic disassembly strategy and the heterogeneous feature fusion design, ensures the lightweight of the model deployment, realizes high-precision reconciliation scoring and label prediction, and balances the engineering practicability and the intelligent level, thereby improving the intelligent processing capability and the business response efficiency of the enterprise industry and financial integration as a whole. BRIEF DESCRIPTION OF DRAWINGS

[0062] The accompanying drawings are used to provide a further understanding of the present application, and constitute a part of the specification, together with the embodiments of the present application, to explain the present application, and do not constitute a limitation on the present application. In the drawings:

[0063] Figure 1 a flowchart of an industry and financial integration method based on deep learning proposed by the present application;

[0064] Figure 2 an industry and financial data coding and semantic mapping flowchart of an industry and financial integration method based on deep learning proposed by the present application;

[0065] Figure 3 a module structure diagram of an industry and financial integration system based on deep learning proposed by the present application. DETAILED DESCRIPTION

[0066] The application will be described in further detail below with reference to the drawings. These drawings are simplified schematic diagrams that only schematically illustrate the basic structure of the application, and thus only show the components relevant to the application.

[0067] Reference Figures 1-2 A deep learning-based industry and finance integration method, comprising the following steps:

[0068] S1, obtaining industry and finance data and performing preprocessing;

[0069] S2, encoding the preprocessed industry and finance data using a double-tower structure, and establishing a semantic mapping relationship between business features and financial features through cross-attention alignment operation, and outputting a matching feature matrix;

[0070] S3, inputting the matching feature matrix into a Transformer-XL network, modeling the context relationship by introducing a gated memory mechanism and a periodic position encoding, and generating a fusion semantic representation with global dependency;

[0071] S4, inputting the fusion semantic representation into a lightweight low-rank adaptive LoRA-BERT network, performing fine-grained alignment and heterogeneous feature fusion in a multi-granularity semantic space, and outputting an industry and finance reconciliation score result and a reconciliation label;

[0072] S5, constructing a supervised training sample, taking the industry and finance reconciliation score result and the reconciliation label as a supervision signal, jointly optimizing the gated memory parameters in the Transformer-XL network and the low-rank mapping parameters in the LoRA-BERT network, and completing double-network joint training;

[0073] S6, integrating and deploying the trained double network, performing parallel coding, context modeling and reconciliation prediction on new industry and finance data, and realizing the whole process of industry and finance integration.

[0074] The application realizes end-to-end modeling from data preprocessing to reconciliation prediction by introducing double-tower structure coding, cross-attention alignment, Transformer-XL context modeling and LoRA-BERT fine-grained fusion, and has the advantages of high automation, strong semantic alignment accuracy and good deployability.

[0075] In the embodiment, the industry and finance data includes business data and financial data, the business data includes warehouse in-out records, material use information and inventory change logs, and the financial data includes voucher records, budget execution data and cost accounting details.

[0076] The application defines the sources and structures of industry and finance data, realizes unified modeling of warehouse in-out records, material use information and voucher data and other multi-source information, and effectively improves the adaptability and generalization ability of business data and financial data fusion.

[0077] In the embodiment, the preprocessing includes field mapping, timestamp alignment, missing value filling and data standardization.

[0078] The application introduces field mapping, time alignment, missing value filling and standardization steps in the data preprocessing stage, guarantees the uniformity of data quality and the controllability of model input, and provides a stable feature basis for downstream deep model training.

[0079] In the embodiment, the S2 specifically includes:

[0080] S21, respectively embedding coding the preprocessed business data and financial data, constructing business feature matrix B=[b1, b2, …, b m ] and financial feature matrix F=[f1, f2, …, f n ], wherein b i represents the i-th business feature vector, f j represents the j-th financial feature vector, m represents the number of business features, and n represents the number of financial features;

[0081] S22, inputting the business feature matrix B and the financial feature matrix F into the cross attention alignment network, establishing the semantic mapping relationship between the business features and the financial features, calculating the semantic correlation matrix M between each pair of business-financial features, and constructing the following attention matching function:

[0082]

[0083] Wherein, M(i,j) represents the element value in the semantic correlation matrix, Q b (i) represents the query vector mapped by the i-th business feature vector b i , K f (j) represents the key vector mapped by the j-th financial feature vector f j , d k represents the vector dimension, γ represents the distance penalty adjustment coefficient, represents the Euclidean distance square between the i-th business feature vector and the j-th financial feature vector, used to quantify the nonlinear semantic difference;

[0084] S23, based on the semantic correlation matrix M, calculating the matching feature matrix Z, and constructing the following feature mapping function:

[0085]

[0086] Wherein, z i represents the i-th matching feature vector, h i represents the financial attention representation after fusion of the i-th business feature vector, and αij is the attention weight normalized by the Softmax function of M(i,j), W h ,W r respectively represent different projection mapping matrices, b z represents a matching bias vector, and sigma represents a nonlinear activation function.

[0087] The application establishes the semantic matching relationship between the business features and the financial features accurately by constructing a double-tower encoding structure and introducing a semantic cross-attention mechanism, generates a high-quality matching feature matrix, and significantly improves the representation ability of the account modeling.

[0088] In the embodiment, the S3 specifically includes:

[0089] S31, input the matching feature matrix Z into the Transformer-XL network, and introduce a gated memory mechanism and a residual connection at each layer to construct a context representation:

[0090]

[0091] wherein, represents the ith matching feature vector of the lth layer, represents the corresponding ith matching feature vector of the previous layer, represents the inherited gated memory state from the historical position, u represents a gating control vector, W1, W2, W u respectively represent different weight matrices, b1, b u is a bias vector, sigma represents a nonlinear activation function, represents a corresponding element multiplication operation, and tanh(·) represents a hyperbolic tangent function.

[0092] S32, add the context representation to the periodic position encoding vector p i to construct a fused semantic representation s i , and the calculation method is:

[0093]

[0094] wherein, s i represents the ith fused semantic representation, sin(·) represents a hyperbolic sine function, omega represents a periodic encoding frequency factor, psi represents a periodic offset term, i represents a position number, and rho represents a normalization function.

[0095] The application introduces a gated memory mechanism and a periodic position encoding based on the Transformer-XL network, enhances the context modeling capability in the fused semantic representation, effectively solves the problems of semantic decay and cross-period dependency modeling in long sequences, and improves the stability of the upstream and downstream tasks.

[0096] In this embodiment, the fused semantic representation constructs a multi-layer residual recursive structure by introducing a gated memory mechanism, performs position-aware context modeling on the matching feature matrix, performs weighted fusion based on the current state and historical memory state in each layer, introduces a periodic position encoding vector, and uses learnable frequency factors and offset terms to construct a periodic dynamic position function to enhance time perception capabilities, maintain semantic continuity and cross-step dependency structure consistency in cross-layer outputs, and finally generate a fused semantic representation sequence.

[0097] The present invention constructs a multi-layer residual recursive network in the context modeling structure, combined with a learnable position encoding function, to further enhance the temporal dynamic perception capability and ensure the consistency and stability of the reconciliation semantic structure in multiple layers of the model.

[0098] In this embodiment, the S4 specifically includes:

[0099] S41, the fusion semantic representation s i Input lightweight low-rank adapted LoRA-BERT network, based on each s i Construct context window vector block C i =[s i-w ,…,s i ,…,s i+w ], and for each s i Perform low-rank attention transformation and construct the following embedding expression:

[0100]

[0101] Among them, v i represents the context-enhanced feature representation of the i-th LoRA transformed feature, α ik is the attention weight normalized by the Softmax function of M(i,k), A u Represents the dimensional mapping matrix in LoRA, A d represents the dimension reduction mapping matrix, w represents the window width, and LayerNorm(·) represents the layer normalization operation;

[0102] S42, each v i The multi-granularity feature fusion structure within the BERT model is input, and three-level fine-grained alignment operations are performed at the sentence level, substructure level, and field level. Heterogeneous projection is performed on each semantic position, and the following heterogeneous alignment expression is constructed:

[0103] e i =σ(W3·v i +W4·f λ(i) +W5·z i +b e );

[0104] lambda(i) = (i * p) mod n

[0105] wherein e i represents the i-th isomerically aligned output vector, lambda(·) represents an index mapping function, f λ(i) represents the lambda(i)-th financial feature vector, p represents an alignment step size, z i represents the i-th matching feature vector, W3, W4, and W5 represent different fusion weight matrices, respectively, b e represents a fusion bias vector, sigma represents a nonlinear activation function, mod is a modulus operator, i represents a position number, and n represents a financial feature quantity.

[0106] S43, integrating all isomerically aligned output vectors e i to generate an industry-finance reconciliation score vector and a reconciliation label, respectively, and defining the following output function:

[0107]

[0108] wherein r i represents the i-th reconciliation score result, t i represents the i-th reconciliation classification label, taking a value of 1 or 2, W s represents a score projection matrix, b s is a score bias vector, W c represents a classification weight matrix, b c is a classification bias, and sigma represents a nonlinear activation function.

[0109] The application realizes high-precision output of reconciliation score results and labels under a lightweight computing framework by introducing a low-rank attention mapping structure into the LoRA-BERT network, cooperating with a multi-granularity semantic alignment and financial feature fusion mechanism, and improving deployment efficiency and reconciliation accuracy.

[0110] In the embodiment, the LoRA-BERT network is constructed based on a pre-trained BERT model, a low-rank parameter pair composed of a trainable dimension-increasing mapping matrix and a dimension-reducing mapping matrix is introduced in each layer of attention calculation, the fused semantic representation is taken as input, the self-attention encoding operation within a local window is performed under the premise of keeping the BERT model backbone parameters unchanged, and the financial feature vector and the matching feature vector are integrated through a residual connection structure to output a fine-grained isomerically fused result for reconciliation score and label generation.

[0111] The application explicitly injects a low-rank adaptive matrix into each layer of attention calculation of the BERT model, realizes semantic refinement and cross-domain fusion under the condition of freezing the pre-trained model parameters, reduces the training cost, and enhances the model migration adaptation ability.

[0112] In this embodiment, S5 specifically includes:

[0113] S51, based on the industry and financial account reconciliation score vector and the reconciliation label, a supervised training sample set is constructed Where q represents the total number of training samples;

[0114] S52, construct a joint loss function Optimize the gating memory parameter set Θ in the Transformer-XL network G And the low-rank mapping parameter set Θ in the LoRA-BERT network L The joint loss expression is defined as follows:

[0115]

[0116] Where, log2(·) represents the logarithmic function, λ represents the score confidence adjustment coefficient, t represents the reconciliation label i The corresponding label vector, V represents the score conversion matrix, b v The score bias vector is represented, μ represents the cross-model regularization penalty coefficient, A G The gating attention residual mapping matrix in the Transformer-XL network is represented, A L The low-rank fusion projection matrix in the LoRA-BERT network is represented, The memory representation generated by the i-th matching feature through the Transformer-XL network is represented by The alignment representation generated by the i-th matching feature through the LoRA-BERT network is represented by The Euclidean norm square of the vector is represented, which is used to quantify the difference;

[0117] S53, using a joint back propagation mechanism, based on minimizing the loss function The gradient descent optimization strategy is used to update the parameter set Θ G And Θ L Using the gradient descent method based on learning rate η to iteratively update, complete the synchronous training of the double network.

[0118] The present application proposes a multi-objective supervised training mechanism for jointly optimizing the Transformer-XL network and the LoRA-BERT network, which combines the reconciliation score and the label supervision signal, effectively realizes the unified collaborative learning of heterogeneous parameter spaces, and enhances the model convergence efficiency and generalization performance.

[0119] Reference Figure 3 A deep learning-based industry and financial integration system, comprising:

[0120] a data processing module, configured to acquire industry and finance data and perform preprocessing;

[0121] an encoding mapping module, configured to encode the preprocessed industry and finance data in a double-tower structure, and establish a semantic mapping relationship between business features and financial features through a cross-attention alignment operation, and output matching feature representations;

[0122] a context modeling module, configured to input the matching feature representations into a Transformer-XL network, and perform context relationship modeling by introducing a gated memory mechanism and a periodic position encoding, and generate fusion semantic representations with global dependencies;

[0123] an alignment fusion module, configured to input the fusion semantic representations into a LoRA-BERT network with lightweight low-rank adaptation, perform fine-grained alignment and heterogeneous feature fusion in a multi-granularity semantic space, and output industry and finance reconciliation score results and reconciliation labels;

[0124] a supervised training module, configured to construct supervised training samples, take the industry and finance reconciliation score results and the reconciliation labels as supervision signals, jointly optimize the gated memory parameters in the Transformer-XL network and the low-rank mapping parameters in the LoRA-BERT network, and complete double-network joint training;

[0125] a deployment application module, configured to integrate and deploy the trained double network, perform parallel encoding, context modeling and reconciliation prediction on newly added industry and finance data, and realize the whole process of industry and finance fusion.

[0126] The present application constructs a complete industry and finance fusion system architecture, forms a closed loop from data processing, feature modeling, context learning, semantic alignment to deployment prediction, realizes intelligent integration of the whole process of industry and finance data from collection to reconciliation, and has high adaptability and practical application value.

[0127] Embodiment 1

[0128] In order to verify the feasibility of the present application in implementation, the present application is applied to the "material warehouse-in and warehouse-out management and finance accounting data fusion" scene of a large comprehensive group, and focuses on intelligent reconciliation optimization deployment for the long-existing problems such as "warehouse-out data lag synchronization, cost accounting deviation, budget execution cannot be closed loop". There are multiple business subsystems in the group, including warehouse-in and warehouse-out management platform, intelligent procurement platform, financial sharing platform and budget control platform, the data structures among the systems are seriously heterogeneous, the fields are not unified and the semantics are inconsistent, which leads to that after business behavior occurs, the financial processing link needs to rely on manual checking of vouchers, screening of abnormal records, supplementary recording or recalculation of cost. The average monthly manual processing of abnormal reconciliation data is more than 1800, accounting for about 13.6%, and the manual intervention time is more than 600 man-hours, which seriously affects the financial closed loop efficiency and budget feedback timeliness.

[0129] In the application process, first of all, through the data processing module, access the group's warehouse account, material use record, inventory change log and financial voucher data, and complete the field uniformity, timestamp alignment, missing completion and other preprocessing operations. Subsequently, through the double tower structure, the business data and financial data are embedded and coded respectively, and the cross attention mechanism is introduced to establish the fine semantic mapping relationship, automatically learning the semantic dependence between "material use-outbound behavior" and "cost entry-financial voucher", completing feature alignment and mapping without manual rule definition. The matching feature matrix constructed based on the mapping is further input into the Transformer-XL network for context modeling, effectively capturing complex semantic dependencies such as cross-cycle budget adjustment and delayed payment, and finally fusing semantic representations into LoRA-BERT structure for multi-granularity alignment, accurately identifying potential abnormalities such as "semantic inconsistency", "amount mismatch" and "budget overrun".

[0130] After the system goes online, in the first month of test data, the business system generates 42138 outbound records, and the financial system generates 39762 voucher data. The system automatically reconciles and matches at a rate of 94.5%, of which 39873 are successfully matched without manual intervention, accounting for 94.6% of the total business data. Among the remaining data that cannot be automatically reconciled, the system identifies 472 abnormal outbound amount deviation problems, 131 outbound time lag problems, and 226 budget unbinding problems, and generates traceable business behavior chains and repair suggestions. Compared with before deployment, the amount of manual review intervention has decreased to 12.8% of the original, and the manual processing time has decreased to less than 100 hours, effectively releasing financial human resources and improving reconciliation efficiency.

[0131] Further statistics show that the average reconciliation score output by the intelligent fusion model is 0.93, with a standard deviation of 0.04 and a stable confidence interval. In the subsequent three-cycle adaptive learning process, the model continuously iterates and optimizes through the feedback learning mechanism, automatically identifies new data structure changes and semantic deviations, and finally stabilizes the reconciliation accuracy rate to more than 96.1%, significantly higher than the 83% level achieved by traditional rule-driven methods. The system feedback delay is controlled within 3 seconds, with good real-time performance and business response ability. The data support table is shown below:

[0132] Table 1 Comparison of performance data of industry and finance reconciliation

[0133]

[0134]

[0135] The above results fully verify the practicability, robustness and business application value of the application in the actual complex business environment, greatly improve the automation degree of industry and finance integration, and provide a technical foundation and empirical evidence for subsequent large-scale deployment.

[0136] The above merely describes preferred specific embodiments of the application, but the protection scope of the application is not limited thereto, and any person skilled in the art can make equivalent replacements or changes to the technical solutions and the inventive concept of the application within the technical scope disclosed by the application, which should be covered within the protection scope of the application.

Claims

1. A business-finance integration method based on deep learning, characterized by: The steps include: S1. Obtain business and financial data and perform preprocessing; S2: Use a dual-tower structure to encode the pre-processed business and financial data, and establish a semantic mapping relationship between business features and financial features through cross-attention alignment operations, and output a matching feature matrix; S3. Input the matching feature matrix into the Transformer-XL network, and model the contextual relationship by introducing a gated memory mechanism and periodic position encoding to generate a fused semantic representation with global dependencies. S4: Input the fused semantic representation into the lightweight, low-rank adapted LoRA-BERT network, perform fine-grained alignment and heterogeneous feature fusion in the multi-granularity semantic space, and output the business and financial reconciliation score results and reconciliation labels; S5. Construct supervised training samples, using the financial reconciliation score and reconciliation labels as supervisory signals, and jointly optimize the gated memory parameters in the Transformer-XL network and the low-rank mapping parameters in the LoRA-BERT network to complete dual-network joint training. S6. Integrate and deploy the trained dual networks to perform parallel encoding, context modeling, and reconciliation prediction on the newly added business and financial data, thus realizing the entire process of business and financial integration.

2. The business-finance integration method based on deep learning according to claim 1 is characterized in that: The business and financial data include business data and financial data. The business data includes warehouse entry and exit records, material requisition information, and inventory change logs. The financial data includes voucher records, budget execution data, and cost accounting details.

3. The business-finance integration method based on deep learning according to claim 1 is characterized in that: The preprocessing includes field mapping, timestamp alignment, missing value filling and data standardization.

4. The method for business-finance integration based on deep learning according to claim 1, characterized in that: The S2 specifically includes: S21, respectively embed the pre-processed business data and financial data to construct the business feature matrix B = [b1, b2, ..., b m ] and the financial characteristic matrix F=[f1,f2,…,f n ], where b i represents the feature vector of the i-th business, f j represents the j-th financial feature vector, m represents the number of business features, and n represents the number of financial features; S22. Input the business feature matrix B and the financial feature matrix F into the cross attention alignment network, establish the semantic mapping relationship between business features and financial features, calculate the semantic relevance matrix M between each pair of business-financial features, and construct the following attention matching function: Among them, M(i,j) represents the element value in the semantic relevance matrix, Q b (i) represents the feature vector b of the i-th service i The mapped query vector, K f (j) represents the j-th financial feature vector f j The key vector obtained by mapping, d k represents the vector dimension, γ represents the distance penalty adjustment coefficient, represents the square of the Euclidean distance between the i-th business feature vector and the j-th financial feature vector, which is used to quantify the nonlinear semantic difference; S23. Based on the semantic relevance matrix M, the matching feature matrix Z is calculated and the following feature mapping function is constructed: Among them, z i represents the i-th matching feature vector, h i represents the financial attention representation after the fusion of the i-th business feature vector, α ij is the attention weight normalized by the Softmax function of M(i,j), W h ,W r Represents different projection mapping matrices, b z represents the matching bias vector, and σ represents the nonlinear activation function.

5. The method for business-finance integration based on deep learning according to claim 1, characterized in that: The S3 specifically includes: S31. Input the matching feature matrix Z into the Transformer-XL network and introduce a gated memory mechanism and residual connection at each layer to construct a contextual representation: in, represents the i-th matching feature vector of the l-th layer, Represents the i-th matching feature vector corresponding to the previous layer, represents the gated memory state inherited from the historical position, u represents the gated control vector, W1, W2, W u Represent different weight matrices, b1, b u is the bias vector, σ represents the nonlinear activation function, ⊙ represents the corresponding element multiplication operation, and tanh(·) represents the hyperbolic tangent function; S32, context representation With the periodic position encoding vector p i Add together to construct a fusion semantic representation s i , calculated as: Among them, s i represents the i-th fused semantic representation, sin(·) represents the hyperbolic sine function, ω represents the periodic encoding frequency factor, ψ represents the periodic offset term, i represents the position number, and ρ represents the normalization function.

6. The method for business-finance integration based on deep learning according to claim 5, characterized in that: The fused semantic representation constructs a multi-layer residual recursive structure by introducing a gated memory mechanism, performs position-aware context modeling on the matching feature matrix, performs weighted fusion based on the current state and the historical memory state in each layer, introduces a periodic position encoding vector, and uses learnable frequency factors and offset terms to construct a periodic dynamic position function to enhance time perception capabilities, maintain semantic continuity and cross-step dependency structure consistency in cross-layer outputs, and finally generate a fused semantic representation sequence.

7. The method for business-finance integration based on deep learning according to claim 1, characterized in that: The S4 specifically includes: S41, the fusion semantic representation s i Input lightweight low-rank adapted LoRA-BERT network, based on each s i Construct context window vector block C i =[s i-w ,…,s i ,…,s i+w ], and for each s i Perform low-rank attention transformation and construct the following embedding expression: Among them, v i represents the context-enhanced feature representation of the i-th LoRA transformed feature, α ik is the attention weight normalized by the Softmax function of M(i,k), A u Represents the dimensional mapping matrix in LoRA, A d represents the dimension reduction mapping matrix, w represents the window width, and LayerNorm(·) represents the layer normalization operation; S42, each v i The multi-granularity feature fusion structure within the BERT model is input, and three-level fine-grained alignment operations are performed at the sentence level, substructure level, and field level. Heterogeneous projection is performed on each semantic position, and the following heterogeneous alignment expression is constructed: yes i =σ(W3·v i +W4·f λ(i) +W5·z i +b e ); λ(i)=(i×p)mod n; Among them, e i represents the i-th heterogeneous alignment output vector, λ(·) represents the index mapping function, f λ(i) represents the λ(i)th financial feature vector, p represents the alignment step, z i represents the i-th matching feature vector, W3, W4, and W5 represent different fusion weight matrices, and b e represents the fusion bias vector, σ represents the nonlinear activation function, mod is the modulus operator, representing the remainder operation, i represents the position number, and n represents the number of financial features; S43, align all heterogeneous output vectors e i Perform integrated representation, generate business and financial reconciliation score vectors and reconciliation labels respectively, and define the following output function: Among them, r i represents the score result of the i-th reconciliation, t i Represents the i-th reconciliation classification label, with a value of 1 or 2, W s represents the score projection matrix, b s is the score bias vector, W c represents the classification weight matrix, b c is the classification bias, σ represents the nonlinear activation function, and argmax is the value of the independent variable when the function takes the maximum value.

8. The method for business-finance integration based on deep learning according to claim 7, characterized in that: The LoRA-BERT network is built based on the pre-trained BERT model. A low-rank parameter pair consisting of a trainable up-dimensionality mapping matrix and a down-dimensionality mapping matrix is ​​introduced in the attention calculation of each layer. The fused semantic representation is taken as input. While keeping the backbone parameters of the BERT model unchanged, the self-attention encoding operation within the local window is performed, and the financial feature vector and the matching feature vector are integrated through the residual connection structure. The fine-grained heterogeneous fusion result is output for reconciliation scoring and label generation.

9. The method for business-finance integration based on deep learning according to claim 1, characterized in that: The S5 specifically includes: S51. Based on the business and financial reconciliation score vector and reconciliation label, construct a supervised training sample set D = Where q represents the total number of training samples; S52. Constructing a joint loss function Optimizing the set of gated memory parameters Θ in the Transformer-XL network G and the low-rank mapping parameter set Θ in the LoRA-BERT network L , define the joint loss expression as follows: Among them, log2(·) represents the logarithmic function, λ represents the score confidence adjustment coefficient, Represents the reconciliation tag t i The corresponding label vector, V represents the score conversion matrix, b v represents the score bias vector, μ represents the cross-model regularization penalty coefficient, and A G represents the gated attention residual mapping matrix in the Transformer-XL network, A L represents the low-rank fusion projection matrix in the LoRA-BERT network, represents the memory representation generated by the Transformer-XL network through the i-th matching feature representation, represents the aligned representation generated by the LoRA-BERT network through the i-th matching feature representation, Represents the square of the Euclidean norm of the vector, used to quantify the difference; S53, using the joint back propagation mechanism, based on minimizing the loss function Gradient descent optimization strategy, jointly update the parameter set Θ G and Θ L , using the gradient descent method based on the learning rate η to iteratively update and complete the synchronous training of the dual networks.

10. A business-finance integration system based on deep learning, executing a business-finance integration method based on deep learning according to any one of claims 1 to 9, characterized in that: include: Data processing module, used to obtain business and financial data and perform pre-processing; The encoding and mapping module is used to encode the pre-processed business and financial data using a dual-tower structure, establish a semantic mapping relationship between business features and financial features through cross-attention alignment operations, and output matching feature representations; The context modeling module is used to input the matching feature representation into the Transformer-XL network, and to model the contextual relationship by introducing a gated memory mechanism and periodic position encoding to generate a fused semantic representation with global dependencies; The alignment and fusion module is used to input the fused semantic representation into the lightweight low-rank adaptive LoRA-BERT network, perform fine-grained alignment and heterogeneous feature fusion in the multi-granularity semantic space, and output the business and financial reconciliation score results and reconciliation labels; The supervised training module is used to construct supervised training samples. It uses the financial reconciliation scoring results and reconciliation labels as supervision signals to jointly optimize the gated memory parameters in the Transformer-XL network and the low-rank mapping parameters in the LoRA-BERT network to complete the joint training of the two networks. The deployment application module is used to integrate and deploy the trained dual networks, perform parallel encoding, context modeling, and reconciliation prediction on the newly added business and financial data, and realize the entire process of business and financial integration.

Images