A data fusion method based on natural language processing

Abstract

This invention relates to the field of electronic digital data processing technology, specifically a data fusion method based on natural language processing. The method includes: acquiring structured numerical and unstructured text data to be processed; mapping the numerical data to descriptive semantic text using instruction templates; inputting the text into a hybrid pre-trained model; correcting vertical domain weights by embedding a low-rank adaptation matrix in the attention layer; achieving depth alignment in a unified vector space; calculating the correlation degree of feature vectors using a fusion attention mechanism; using the global semantics of a generative architecture as the query vector and the fine-grained features of a discriminative architecture as the key vector; dynamically adjusting the optimal feature weights using policy gradients through reinforcement learning and with feature fusion accuracy as the reward function; generating target feature vectors through nonlinear coupling based on the weights; introducing a continuous learning mechanism to capture contextual drift and calibrate bias parameters in real time, outputting enhanced features. This invention improves the depth and timeliness of heterogeneous data fusion.

Description

Technical Field

[0001] This invention relates to the field of electronic digital data processing technology, specifically a data fusion method based on natural language processing. Background Technology

[0002] With the digital transformation of modern banking, risk assessment before lending has shifted from simply reviewing financial statements to a comprehensive analysis of multi-source, dynamic data. When processing unstructured information such as corporate annual reports and public opinion texts, the extreme heterogeneity of data dimensions makes achieving deep integration of textual semantics and numerical indicators a core requirement for modern bank risk management departments to improve decision-making accuracy.

[0003] Current digital data processing technologies primarily rely on pre-trained language models to extract features from text and combine text vectors with numerical features using traditional linear weighting or concatenation methods. However, when processing lengthy financial documents, existing technologies often struggle to balance the relationship between local specialized entities and the global semantic context, resulting in insufficient model adaptability to vertical domains. Furthermore, traditional models are typically in a static prediction mode, unable to adjust internal parameters in real time according to changes in the macroeconomic environment. They exhibit significant lag when facing financial context drift and lack effective feedback mechanisms to dynamically optimize the contribution weights of cross-modal features.

[0004] This invention aims to solve the problems in existing technologies, such as insufficient fusion of multi-source data, difficulty in models to capture the nonlinear coupling relationship between financial fluctuations and text semantics, and inability to cope with feature degradation caused by context drift, thereby achieving high-precision enhanced feature output with timely calibration capabilities.

[0005] To address this, a data fusion method based on natural language processing is proposed. Summary of the Invention

[0006] The purpose of this invention is to provide a data fusion method based on natural language processing, which improves the depth and timeliness of heterogeneous feature fusion through cross-modal semantic alignment and dynamic weight evolution.

[0007] To achieve the above objectives, the present invention provides the following technical solution: A data fusion method based on natural language processing includes: Acquire structured numerical data and unstructured text data to be processed; map the structured numerical data into descriptive semantic text using a preset instruction template; Descriptive semantic text and unstructured text data are input into a hybrid pre-trained model; by embedding a low-rank adaptation matrix in the attention layer of the hybrid pre-trained model, vertical domain weight correction is performed on the hybrid pre-trained model, and deep alignment of different modal features is achieved in a unified vector space. The correlation between aligned feature vectors is calculated using a fusion attention mechanism; the global semantic representation from the generative architecture is used as the query vector, and the fine-grained features from the discriminative architecture are used as the key vector and value vector. Through reinforcement learning, the optimal feature weights between the query vector and the key-value vector are dynamically adjusted by using the feature fusion accuracy as the reward function and the policy gradient update. Based on the optimal feature weights, local features are nonlinearly coupled with the global semantic background to generate a fused target feature vector. A continuous learning mechanism is introduced to capture the contextual drift of the target feature vector in the time dimension, and the bias parameters of the hybrid pre-trained model are calibrated in real time according to the drift amount, outputting enhanced features for decision support.

[0008] Preferably, mapping the structured numerical data into descriptive semantic text using a preset instruction template includes: dividing the acquired structured numerical data into multiple dimensional indicators according to attributes, and normalizing them according to preset benchmark values ​​of each dimensional indicator; matching corresponding degree descriptive words from a preset financial dictionary enhancement unit according to the interval of the normalized values; extracting the prompt word skeleton from the instruction template, filling the names of the dimensional indicators and the matched degree descriptive words into the corresponding slots of the prompt word skeleton to generate semantic text fragments; concatenating the semantic text fragments with the corresponding indicator values, and adding preset role guidance prefixes to form descriptive semantic text.

[0009] Preferably, the hybrid pre-trained model includes a deep encoding layer, an autoregressive inference layer, an attention layer, an attention interaction layer, and a semantic calibration output layer: the deep encoding layer consists of multi-layer transformer encoding operators, used for fine-grained semantic modeling of unstructured text; extracting named entities and financial terminology from the text and outputting local feature vectors; the autoregressive inference layer is based on a generative transformer architecture, used for logical chain inference of long texts; parsing implicit risk factors in the text and generating query vectors reflecting the global context; the attention layer internally embeds low-rank adaptation matrices in parallel; without changing the model's pre-training parameters, it captures the non-linear coupling relationship between structured numerical indicators and unstructured text through low-order matrix operations, achieving feature alignment in the vertical domain; the attention interaction layer uses a fusion attention mechanism for weight allocation; it calculates the cross-correlation degree between the query vector output by the autoregressive inference layer and the key-value vector output by the deep encoding layer, and dynamically allocates feature weights; the semantic calibration output layer is used to execute a continuous learning mechanism and bias parameter calibration; it dynamically adjusts the final output enhanced feature vector according to the contextual drift of features in the time dimension.

[0010] Preferably, the step of achieving feature depth alignment within the attention layer includes: keeping the original pre-trained weight matrix of the attention layer in the hybrid pre-trained model in an untrainable state, and constructing an adaptation branch composed of low-order matrices next to the pre-trained weight matrix; inputting the input modal features in parallel into the pre-trained weight matrix and the adaptation branch, and weighting and merging the low-order transformation features output by the adaptation branch with the original features output by the pre-trained weight matrix to obtain the corrected enhanced feature representation; and by performing domain fine-tuning on the low-order matrices in the adaptation branch, capturing the correlation between the numerical fluctuation patterns in descriptive semantic text and the semantics of unstructured text, so that features of different dimensions are mapped to the same high-dimensional vector space.

[0011] Preferably, the step of calculating the correlation between aligned feature vectors using a fusion attention mechanism includes: performing a dot product operation on the query vector output by the generative architecture and the key vector output by the discriminative architecture; normalizing and scaling the dot product result; calculating the attention weight distribution of each feature using a hierarchical function; and using the attention weight distribution to perform a weighted summation on the value vector output by the discriminative architecture to obtain the correlation.

[0012] Preferably, the step of dynamically adjusting the feature weights between the query vector and the key vector using reinforcement learning includes: taking the query vector, key vector, and preliminarily calculated attention weight distribution in the current time series as the state input of reinforcement learning; outputting weight adjustment actions according to the current state; applying a nonlinear offset operator to the attention weight distribution to achieve dynamic fine-tuning of the contribution of global semantics and local fine-grained features; calculating the F1 score gain between the output result of the hybrid pre-trained model and the preset label, feeding it back to reinforcement learning as a reward signal; and using the policy gradient algorithm to optimize the probability distribution of the adjustment actions to obtain the optimal feature weights.

[0013] Preferably, the step of generating the fused target feature vector includes: using the optimal feature weights dynamically adjusted by reinforcement learning, performing scalar multiplication operations on the fine-grained feature vector and the global semantic representation respectively to obtain the reconstructed local salient features and global semantic background features; concatenating the reconstructed local salient features and global semantic background features into vectors to construct an original fused feature space containing cross-modal interaction information; and performing high-dimensional projection mapping on the concatenated vector through a preset nonlinear activation function to generate the target feature vector by compressing redundant information and enhancing the feature response of key risk dimensions.

[0014] Preferably, the continuous learning mechanism and the steps for enhancing feature output include: real-time collection of external macroeconomic data and industry sentiment information; extraction of a reference vector for the current time series using the hybrid pre-trained model; determination of context drift by measuring the distribution difference between the target feature vector and the reference vector on the time axis; construction of a calibration function based on the context drift; online micro-incremental adjustment of the bias operator of the output layer of the hybrid pre-trained model to compensate for semantic recognition deviations caused by changes in the macro environment; inputting the calibrated feature vector into a preset risk warning operator; combining the topological association strength in the knowledge graph for feature enhancement; and outputting enhanced features.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention transforms numerical indicators into descriptive text through instruction templates and cleverly embeds a low-rank adaptation matrix into the attention layer of a hybrid model. Compared to traditional simple feature concatenation, this approach effectively captures the implicit correlation between the fluctuation patterns of financial values ​​and the semantics of lengthy texts without compromising the general semantic capabilities of large-scale pre-trained models. This deep cross-modal interaction breaks down the representational boundaries between data, enabling the model to accurately identify complex risk signals such as "operational uncertainty" within a unified vector space. In practical scenarios such as SME credit assessment, the F1 score can be improved compared to traditional models.

[0016] 2. This invention innovatively combines reinforcement learning with a fusion attention mechanism, using F1 score gain as the reward signal to dynamically optimize the weight allocation between the global semantics of the generative architecture and the fine-grained features of the discriminative architecture. This closed-loop feedback system changes the static nature of traditional attention mechanism weight calculation, enabling the model to automatically identify which features are more critical in the current decision based on the state input at different time sequences. Through continuous iteration of policy gradients, the model can efficiently extract the most discriminative target feature vectors from massive heterogeneous data, enhancing the sensitivity to identify risks of collaborative fraud and implicit associations.

[0017] 3. This invention captures the contextual drift of target features over time in real time through a continuous learning mechanism and performs online micro-incremental adjustments to the bias parameters based on the amount of drift. This mechanism effectively solves the model degradation problem caused by fluctuations in financial data with macroeconomic cycles, enabling the system to automatically compensate for semantic recognition biases caused by "contextual drift" in real time. Combined with the enhanced topological associations of the knowledge graph, the final output features not only include static cross-modal alignment information but also incorporate time-sensitive risk correction terms. This dynamic calibration capability provides banks with more forward-looking risk warning support, ensuring the fairness and automation of credit decisions. Attached Figure Description

[0018] Figure 1 This is a flowchart of a data fusion method based on natural language processing proposed in this invention; Figure 2 This is a flowchart of a data fusion method based on natural language processing proposed in this invention. Figure 3 This is a flowchart of the method for obtaining optimal feature weights according to the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Example 1: Please see Figures 1 to 3 This invention provides a data fusion method based on natural language processing, the technical solution of which is as follows: A data fusion method based on natural language processing, such as Figures 1-2 As shown, it includes: Acquire structured numerical data and unstructured text data to be processed; map the structured numerical data into descriptive semantic text using a preset instruction template; Descriptive semantic text and unstructured text data are input into a hybrid pre-trained model; by embedding a low-rank adaptation matrix in the attention layer of the hybrid pre-trained model, vertical domain weight correction is performed on the hybrid pre-trained model, and deep alignment of different modal features is achieved in a unified vector space. The correlation between aligned feature vectors is calculated using a fusion attention mechanism; the global semantic representation from the generative architecture is used as the query vector, and the fine-grained features from the discriminative architecture are used as the key vector and value vector. Through reinforcement learning, the optimal feature weights between the query vector and the key-value vector are dynamically adjusted by using the feature fusion accuracy as the reward function and the policy gradient update. Based on the optimal feature weights, local features are nonlinearly coupled with the global semantic background to generate a fused target feature vector. A continuous learning mechanism is introduced to capture the contextual drift of the target feature vector in the time dimension, and the bias parameters of the hybrid pre-trained model are calibrated in real time according to the drift amount, outputting enhanced features for decision support.

[0021] Furthermore, the structured numerical data is mapped into descriptive semantic text using a preset instruction template, including: dividing the acquired structured numerical data into multiple dimensional indicators according to attributes, and normalizing them according to preset benchmark values ​​of each dimensional indicator; matching corresponding degree descriptive words from a preset financial dictionary enhancement unit according to the interval of the normalized values; extracting the prompt word skeleton from the instruction template, filling the names of the dimensional indicators and the matched degree descriptive words into the corresponding slots of the prompt word skeleton to generate semantic text fragments; concatenating the semantic text fragments with the corresponding indicator values, and adding preset role guidance prefixes to form descriptive semantic text.

[0022] The normalization process is calculated using a preset baseline function, as shown in the following formula: in, The normalized scalar value. To obtain the original structured numerical data, and These represent the upper and lower industry benchmarks for the corresponding dimension indicators within a preset benchmark value library. The preset benchmark value library is a set of numerical boundaries determined by collecting historical credit sample data from the same industry and using percentile statistics. The description of the degree of matching based on the normalized numerical interval includes: dividing the interval [0, 1] into several sub-intervals, each sub-interval being mapped using a mapping function. Corresponding to a specific degree descriptor For example, when At that time, the matching degree description is "extremely high" or "significantly better than the industry average". The preset financial dictionary enhancement unit is a mapping matrix constructed based on professional terms in Basel III regulatory guidelines and combined with word frequency statistics of loan officers' on-site investigation notes.

[0023] Before performing normalization, the original values... Perform truncation processing, that is, when Greater than the industry upper limit benchmark At that time, take ;when Less than the lower limit benchmark At that time, take Ensure the normalized scalar Strictly locked within the interval [0, 1] to ensure the mapping function It can retrieve unique degree descriptors from a pre-defined financial dictionary mapping matrix.

[0024] The prompt word skeleton is implemented by defining a semantic template function, the logic of which is as follows: ; in, For the generated semantic text fragments; The indicator name is a semantic label derived from the attribute fields in structured numerical data. The degree descriptor is derived from the aforementioned normalized values. Qualitative semantic identifiers obtained after interval mapping; and These are the preset skeleton prefix and skeleton suffix, used to limit the descriptive context of the indicator; It is a semantic connection operator used to automatically match conjunctions (such as "behave as", "be at", or "be at a relatively high level") based on financial grammar logic.

[0025] In the specific implementation process, the system parses the slot identifier in the instruction template to... and As an input to the template function as an argument. For example, when For "current ratio", When the value is "significantly higher than the industry average", the function uses a preset prompt skeleton to indicate "the subject's..." Current level of stability "Slot filling is performed, and natural language splicing technology is used to generate text fragments with clear logical relationships."

[0026] The preset role guidance prefix is ​​defined by the task anchor operator. The above text fragments are projected into a task context space that the hybrid pre-trained model can understand. The prefix is ​​obtained by pre-collecting at least 1000 historical credit approval reports, using a text clustering algorithm to extract guiding phrases with a frequency higher than 20% (such as "According to flow analysis:"), and using these as the starting identifier for model input. In this way, the generated descriptive semantic text not only contains the features of the original numerical values ​​but also possesses a language structure consistent with financial logic, thus providing standardized semantic input for subsequent deep alignment of structured indicators and unstructured text in the attention interaction layer.

[0027] This invention achieves deep conversion between numerical values ​​and semantics through instruction template technology. By normalizing and applying industry benchmark libraries, heterogeneous numerical values ​​are transformed into scalars with a dimension of 1. Combined with an enhanced dictionary built based on Basel III criteria, the representational boundary between structured data and unstructured text is eliminated. Using semantic template functions and task anchor operators, numerical features are precisely projected into a task context space that the model can understand, preserving not only the objectivity of the data but also endowing it with a linguistic structure consistent with financial logic. This provides high-quality aligned input to the attention interaction layer, enhancing the model's sensitivity to identifying nonlinear coupling relationships in risk signals.

[0028] Furthermore, the hybrid pre-trained model includes a deep encoding layer, an autoregressive inference layer, an attention layer, an attention interaction layer, and a semantic calibration output layer: the deep encoding layer consists of multi-layer transformer encoding operators, used for fine-grained semantic modeling of unstructured text; extracting named entities and financial terminology from the text and outputting local feature vectors; the autoregressive inference layer is based on a generative transformer architecture, used for logical chain inference of long texts; parsing implicit risk factors in the text and generating query vectors reflecting the global context; the attention layer internally embeds low-rank adaptation matrices in parallel; without changing the model's pre-training parameters, it captures the non-linear coupling relationship between structured numerical indicators and unstructured text through low-order matrix operations, achieving feature alignment in the vertical domain; the attention interaction layer utilizes a fusion attention mechanism for weight allocation; it calculates the cross-correlation degree between the query vector output by the autoregressive inference layer and the key-value vector output by the deep encoding layer, and dynamically allocates feature weights; the semantic calibration output layer is used to execute a continuous learning mechanism and bias parameter calibration; it dynamically adjusts the final output enhanced feature vector according to the contextual drift of features in the time dimension.

[0029] The basic architecture of the hybrid pre-trained model is a combination of generative and discriminative models based on the Transformer structure. Specifically, the autoregressive inference layer uses the GPT-3 architecture as the backbone network to perform global logical inference; the deep encoding layer uses the BERT architecture as the backbone network to perform fine-grained entity extraction and semantic modeling.

[0030] The deep encoding layer and autoregressive inference layer employ a dual-stream parallel input architecture. The acquired unstructured text data and descriptive semantic text derived from structured numerical values ​​serve as the model's basic inputs. In the deep encoding layer, the descriptive semantic text is concatenated with the unstructured text through a pre-defined masking mechanism, enabling the encoding operator to explicitly capture the lexical relationships between numerical indicators and specific textual entities (such as financial items or risk warnings) when extracting local feature vectors. In the autoregressive inference layer, the descriptive semantic text is injected as cue words, guiding the model to establish a global numerical context within lengthy logical reasoning.

[0031] The low-rank adaptation matrix in the attention layer is obtained by adjusting the query weights in the attention mechanism. Key weights The implementation using low-rank decomposition has the following low-order matrix operation logic: ; in, For the frozen pre-trained weight matrix, and For a trainable low-order matrix, rank And its dimension is 1. This is the aligned feature vector output by this layer. Through this structure, the model can capture the vertical domain nonlinear bias of numerical indicators in the semantic space without compromising general semantics.

[0032] The cross-association calculation formula performed by the attention interaction layer is defined as follows: ; in, This is the query vector output by the autoregressive inference layer. The key vector is the output of the deep coding layer. Scaling factor This is a dynamic bias correction term output by the reinforcement learning agent based on the feature fusion accuracy feedback. This layer uses dot product similarity measurement to accurately map the global context to the key feature dimensions of local entities.

[0033] The context drift in the semantic calibration output layer The calculation logic is as follows: ; in, The target feature distribution at the current moment, This is the baseline distribution for the previous time series. When the drift... Exceeding the preset threshold At that time, the calibration mechanism is activated, through the gradient adjustment factor. Bias parameters of the output layer Perform online micro-incremental updates, i.e. This process ensures that the output enhanced feature vectors possess semantic consistency and decision robustness across different macroeconomic cycles and contexts.

[0034] This invention achieves collaborative modeling of fine-grained entities and global logic through a dual-path architecture of deep encoding and autoregressive inference. Utilizing parallel computation of low-rank adaptation matrices, it accurately captures nonlinear biases in vertical domains without sacrificing general semantics. Simultaneously, it introduces reinforcement learning dynamic correction terms to optimize attention allocation and establishes a drift monitoring mechanism, achieving online adaptive calibration of bias parameters through gradient adjustment factors. This not only improves the fusion accuracy of heterogeneous data in complex contexts but also ensures the model's decision robustness in dynamic environments.

[0035] Furthermore, the step of achieving feature depth alignment within the attention layer includes: maintaining the original pre-trained weight matrix of the attention layer in the hybrid pre-trained model in an untrainable state, and constructing an adaptation branch composed of low-order matrices next to the pre-trained weight matrix; inputting the input modal features in parallel into the pre-trained weight matrix and the adaptation branch, and weighting and merging the low-order transformation features output by the adaptation branch with the original features output by the pre-trained weight matrix to obtain the corrected enhanced feature representation; and by performing domain fine-tuning on the low-order matrices in the adaptation branch, capturing the correlation between numerical fluctuation patterns in descriptive semantic text and unstructured text semantics, so that features of different dimensions are mapped to the same high-dimensional vector space.

[0036] The original pre-trained weight matrix within the attention layer includes the query weight matrix. Key weight matrix Sum weight matrix The construction and weighted merging process of the adapted branches follows the following mathematical definition: ; in, This is the original pre-trained weight matrix in a non-trainable state. The input modal feature vector, and This is the low-order matrix that constitutes the adaptive branch. The preset scaling smoothing factor is calculated as follows: ,in For the preset rank, This is a constant determined based on the vertical domain characteristic distribution, used to adjust the correction intensity. The initialization logic of the low-order matrix is ​​as follows: [The text abruptly ends here, likely due to an incomplete sentence or missing information.] A Gaussian distribution is used for random initialization to ensure diversity in feature projections during the initial stage; for the matrix... All-zero initialization is used to ensure that the output of the adaptation branch is zero at the beginning of fine-tuning training, thereby maintaining the stability of the original features.

[0037] During fine-tuning, the alignment of the same high-dimensional vector space is achieved by minimizing the cross-modal contrastive loss function. accomplish: ; in, and These represent the descriptive semantic text projection and the unstructured text semantic projection after transformation by the enhanced feature representation, respectively. Through this alignment mechanism, the low-order matrix can collaboratively learn the bias of structured numerical values ​​in the semantic space, thereby achieving a deep nonlinear coupling between numerical fluctuation patterns and textual context.

[0038] This invention achieves precise correction of vertical domain features by constructing low-order adaptation branches in the attention layer. Employing a bypass structure with all-zero initialization ensures model stability during initial fine-tuning (output is zero) while flexibly adjusting the correction intensity using a scaling smoothing factor. Combined with the minimization constraint of the cross-modal contrastive loss function, it forces the fluctuation patterns of structured numerical values ​​and the semantics of unstructured text to achieve deep alignment within the same high-dimensional vector space. This design not only reduces the computational cost of vertical domain adaptation but also enhances the model's ability to capture deep nonlinear coupling relationships between multimodal features.

[0039] Furthermore, the step of calculating the correlation between aligned feature vectors using the fusion attention mechanism includes: performing a dot product operation on the query vector output by the generative architecture and the key vector output by the discriminative architecture; normalizing and scaling the dot product result; calculating the attention weight distribution of each feature using a hierarchical function; and using the attention weight distribution to perform a weighted summation on the value vector output by the discriminative architecture to obtain the correlation.

[0040] The normalization scaling process and the calculation of the attention weight distribution follow the following mathematical logic: in, For query vector, The result of the dot product operation is the key vector. via scaling factor To address the gradient vanishing problem caused by dimensionality growth, The dimension of the key vector. The hierarchical function. A preset temperature regulation factor has been introduced. It is obtained by using a grid search algorithm on the validation set to control the smoothness of the weight distribution, enabling the model to adaptively focus on key features according to the semantic depth of different levels.

[0041] The specific steps for weighted summation and obtaining the correlation degree are as follows: in, The value vector output by the discriminative architecture. This refers to the degree of correlation. Physically, this degree of correlation is represented as a high-dimensional fused feature vector that aggregates global semantic background and local numerical features.

[0042] This invention achieves precise aggregation of cross-architecture features through a fusion attention mechanism. By utilizing a scaling factor to offset gradient interference caused by dimensionality growth, and combining this with a hierarchical function incorporating a temperature-adjusting factor, the model can adaptively optimize weight distribution based on semantic depth, ensuring focus on key features. The resulting association degree exists as a high-dimensional fusion feature vector, deeply coupling global context and local numerical features, providing a semantic alignment foundation for subsequent dynamic weight optimization in reinforcement learning.

[0043] Furthermore, such as Figure 3 As shown, the steps for dynamically adjusting the feature weights between the query vector and the key vector using reinforcement learning include: taking the query vector, key vector, and the initially calculated attention weight distribution as the state input for reinforcement learning; adjusting the weights according to the current state; applying a nonlinear offset operator to the attention weight distribution to achieve dynamic fine-tuning of the contribution of global semantics and local fine-grained features; calculating the F1 score gain between the output of the hybrid pre-trained model and the preset label, feeding it back to reinforcement learning as a reward signal; and using the policy gradient algorithm to optimize the probability distribution of the adjustment action to obtain the optimal feature weights.

[0044] The status input The construction logic is as follows: This involves concatenating the query vector, key vector, and preliminary attention distribution into a single vector.

[0045] The nonlinear offset operator The weight adjustment action under the influence is defined as follows: in, To reinforce the offset vector output by the learning agent, it is obtained by performing a nonlinear transformation on the state space through a multilayer perceptron, which is used to correct the residual of the original weight distribution. To obtain the optimal feature weights, Through multilayer perceptron Capture state space The nonlinear features in the data are analyzed, and an offset vector consistent with the original weight dimension is generated.

[0046] Nonlinear offset operator The specific implementation is as follows: Construct a residual network containing two fully connected layers, with an input dimension of... , The dimension of the vector concatenated with the initial weights. The first layer uses the ReLU activation function to capture the nonlinear perturbation between the semantics and numerical fluctuations of financial text; the second layer outputs an offset vector with the same dimension as the attention matrix. By superimposing this offset vector onto the original Softmax output, the model is forced to increase the attention weight to the dimension related to "financial deviation" as the F1 score decreases, thereby achieving task-oriented feature focusing.

[0047] The reward signal R is calculated as follows: ,in To calculate the F1 score of the hybrid pre-trained model in the current batch, we guide the model towards high-precision fusion by calculating its gain relative to the previous time series.

[0048] The optimization process of the policy gradient algorithm follows the following formula: in, For parameterized strategies, it represents the state The following adjustment action was taken. The probability distribution. Parameters are continuously optimized through gradient ascent. This increases the probability of weight adjustment actions that can generate positive F1 gain, thereby enabling intelligent fine-tuning of the contribution of global semantics and local fine-grained features.

[0049] This invention defines a nonlinear offset operator. This mechanism nonlinearly fuses the initial attention weights with the residual offsets generated by reinforcement learning. It utilizes a multilayer perceptron to capture deep features in the state space, intelligently fine-tuning the contributions of global semantics and local features. Renormalization via the Softmax operator ensures the logical completeness of the weight allocation, improving the model's feature fusion accuracy and decision sensitivity in dynamic environments.

[0050] Further, the steps for generating the fused target feature vector include: using the optimal feature weights dynamically adjusted by reinforcement learning, performing scalar multiplication operations on the fine-grained feature vector and the global semantic representation respectively to obtain the reconstructed local salient features and global semantic background features; concatenating the reconstructed local salient features and global semantic background features into vectors to construct an original fused feature space containing cross-modal interaction information; and performing high-dimensional projection mapping on the concatenated vector through a preset nonlinear activation function to generate the target feature vector by compressing redundant information and enhancing the feature response of key risk dimensions.

[0051] The specific logic of the scalar multiplication operation is as follows: using the first weight component in the optimal feature weights. With the second weighted component Calculate the local saliency features respectively With global semantic background features ;in, For fine-grained feature vectors, It is a global semantic representation and satisfies .

[0052] The mathematical expression for the high-dimensional projection mapping is defined as follows: in, This represents the increased-dimensional vector obtained by concatenating the reconstructed local salient features with the global semantic background features. This is a preset projection weight matrix; This is the bias vector. The preset nonlinear activation function... use The function, its formula is ;in These are learnable scaling parameters used to compress redundant information and enhance the feature responses of key dimensions through a nonlinear gating mechanism.

[0053] The projection weight matrix The method for obtaining the target feature space is as follows: during the end-to-end training phase of the hybrid pre-trained model, the target feature space is automatically learned by minimizing the cross-entropy loss function of the target task and using the backpropagation algorithm. Through this projection transformation, the original fused feature space is mapped to a target feature space with higher feature density and stronger nonlinear expressive power, ultimately generating a target feature vector, which provides standardized feature input for subsequent context drift detection.

[0054] This invention achieves deep coupling of multimodal features through scalar reconstruction guided by optimal feature weights and nonlinear projection mapping. An original fusion space is constructed by vector concatenation, and information compression and risk dimension enhancement are performed in a high-dimensional space using a learnable projection weight matrix, effectively eliminating feature redundancy. This mechanism enhances the nonlinear expressive power of the target feature vector, providing a feature foundation for accurately capturing financial fluctuations and semantic relationships in complex environments.

[0055] Furthermore, the continuous learning mechanism and the steps for enhancing feature output include: real-time collection of external macroeconomic data and industry sentiment information; extraction of the reference vector for the current time series using the hybrid pre-trained model; determination of context drift by measuring the distribution difference between the target feature vector and the reference vector on the time axis; construction of a calibration function based on the context drift; online micro-incremental adjustment of the bias operator of the output layer of the hybrid pre-trained model to compensate for semantic recognition deviations caused by changes in the macro environment; inputting the calibrated feature vector into a preset risk warning operator; combining the topological association strength in the knowledge graph for feature enhancement; and outputting enhanced features.

[0056] The reference vector The extraction logic is as follows: the collected macroeconomic indicators (such as interest rates and inflation rates) and industry public opinion keywords are projected into the same semantic space through the hybrid pre-trained model, and the average value of features within the sliding time window is taken as the benchmark representation.

[0057] The distributional difference is measured using a weighted combination of cosine similarity and Euclidean distance, with its contextual drift being... The calculation formula is defined as follows: in, For the target feature vector, These are preset weighting coefficients used to balance the contributions of directional deviation and Euclidean distance to the drift amount. Weighting coefficients The value range is [0.4, 0.6], and in this embodiment, it is preferred to... The coefficient is determined by comparing the model's performance during historical context switching periods: when drastic changes in industry policies lead to changes in semantic meaning, the cosine similarity weight is increased to capture directional drift; when macroscopic numerical fluctuations are drastic but semantics remain unchanged, the Euclidean distance weight is increased to capture magnitude deviation.

[0058] The online incremental adjustment of the bias operator b is achieved through a preset calibration function f(D), the specific recursive formula of which is: in, The preset learning rate step size is obtained through an online stochastic gradient descent algorithm.

[0059] The topological association strength is obtained by calculating the sum of the shortest path weights between the target entity and its associated risk nodes in the knowledge graph. The resulting enhanced feature vector... Defined as a concatenation mapping between calibrated vectors and topological features, its physical meaning is that while maintaining the original feature logic, macro-contextual correction terms and knowledge graph risk contagion terms are superimposed.

[0060] This invention solves the logical hedging problem of distance metrics in multidimensional space by reconstructing the computational model of contextual drift. By utilizing the co-directional weighting of cosine distance and normalized Euclidean exponent terms, linear perception of feature direction and intensity drift is achieved. This improved metric mechanism eliminates fundamental logical errors, ensuring that the drift can stably reflect external environmental fluctuations. It provides a reliable numerical basis for online adaptive calibration of bias parameters and significantly improves the long-term robustness of decision features under extreme economic cycles.

[0061] This invention transforms structured numerical values ​​into descriptive semantic text through instruction templates, breaking down representational barriers between heterogeneous data. A low-rank adaptation matrix is ​​used at the attention layer to correct vertical domain weights, ensuring deep alignment of multimodal features within a unified vector space. A reinforcement learning-based dynamic weight adjustment mechanism is introduced to optimize the contribution of global semantics and fine-grained features, guided by feature fusion accuracy, thereby enhancing the expressive power of the fused features. A continuous learning mechanism is combined to capture contextual drift and calibrate bias parameters in real time, effectively compensating for semantic recognition biases caused by changes in the macro-environment, ensuring the long-term robustness and timeliness of feature output.

[0062] Example 2 This invention provides a data fusion method based on natural language processing. The following uses credit risk assessment before bank lending as a specific application scenario to describe the implementation process of this invention in detail.

[0063] In a credit assessment of a manufacturing company, the system obtains structured numerical data (such as current ratio and debt-to-equity ratio) and unstructured text data (such as the company's annual reports and recent industry news and public opinion). The system first performs truncation on the raw data, restricting it to values ​​defined in the industry benchmark database. and Within the boundary, the scalar value is calculated using a normalization function. Subsequently, according to The sub-interval in question is matched with descriptive terms such as "extremely high liquidity" or "debt level significantly better than industry average" from the enhanced financial dictionary unit based on Basel III regulatory principles. Using semantic template functions, the indicator name is... Degree description of matching The information is filled into the prompt word skeleton, and role-guided prefixes extracted by text clustering algorithms, such as "According to flow analysis, it can be seen that:", are added to ultimately form logically rigorous descriptive semantic text.

[0064] The descriptive semantic text and the original unstructured text data are fed in parallel into a hybrid pre-trained model. In the deep encoding layer, the model extracts local feature vectors of financial terminology through multi-layer transform operators; in the autoregressive inference layer, a generative architecture is used to parse the logical chains in long documents, generating query vectors Q that reflect the global context. To achieve feature correction for vertical domains, an adaptive branch is enabled in the attention layer, maintaining the pre-trained weight matrix... While not trainable, it utilizes initialization with a Gaussian distribution. With all-zero initialization Low-order matrix operations are performed. This is achieved by minimizing the cross-modal contrastive loss function. This allows the numerical fluctuation patterns of structured indicators to achieve deep alignment with the textual context within the same high-dimensional vector space.

[0065] The feature correlation is calculated using a fusion attention mechanism, and the global query vector output by the autoregressive inference layer is used. Key vectors output by the deep coding layer Perform dot product operations and introduce a temperature adjustment factor obtained from grid search. This controls the smoothness of the weight distribution. Then, the query vector, key vector, and initial attention weight distribution for the current time series are concatenated into the state input. The reinforcement learning agent uses a nonlinear offset operator... An offset is applied to the original distribution to dynamically fine-tune the contribution weights of global semantics and local fine-grained features. When the risk assessment result output by the model produces a positive gain with the F1 score of the preset label, the policy gradient algorithm optimizes and adjusts the probability distribution of actions to determine the optimal feature weights.

[0066] Scalar multiplication and vector concatenation are performed on local salient features and global semantic background features using optimal weights. The concatenated, increased-dimensional vector is then subjected to high-dimensional projection mapping using a pre-defined Swish non-linear activation function to compress redundant information and enhance the feature response of key risk dimensions, generating the target feature vector. To address changes in the macroeconomic environment, a continuous learning mechanism collects real-time macroeconomic indicators (such as interest rate fluctuations) and industry sentiment, extracting reference vectors. The context drift was calculated by weighting the cosine similarity and Euclidean distance in the same direction. Based on this, the bias operator b of the output layer is calibrated incrementally using an online stochastic gradient descent algorithm. Finally, the calibrated vector is input into the risk warning operator and enhanced by combining the sum of the shortest path weights between the enterprise and other risk subjects in the knowledge graph, ultimately outputting an enhanced feature vector to support bank lending decisions.

[0067] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A data fusion method based on natural language processing, characterized in that, include: Acquire structured numerical data and unstructured text data to be processed; map the structured numerical data into descriptive semantic text using a preset instruction template; Descriptive semantic text and unstructured text data are input into a hybrid pre-trained model; By embedding a low-rank adaptation matrix in the attention layer of the hybrid pre-trained model, vertical domain weight correction is performed on the hybrid pre-trained model, achieving alignment of different modal features in a unified vector space. The correlation between aligned feature vectors is calculated using a fusion attention mechanism; the global semantic representation from the generative architecture is used as the query vector, and the fine-grained features from the discriminative architecture are used as the key vector and value vector. Through reinforcement learning, the optimal feature weights between the query vector and the key-value vector are dynamically adjusted by using the feature fusion accuracy as the reward function and the policy gradient update. Based on the optimal feature weights, local features are non-linearly coupled with the global semantic background to generate a fused target feature vector; [the following is introduced] The continuous learning mechanism captures the contextual drift of the target feature vector over time and calibrates the bias parameters of the hybrid pre-trained model in real time based on the drift amount, outputting enhanced features for decision support.

2. The data fusion method based on natural language processing according to claim 1, characterized in that, The structured numerical data is mapped into descriptive semantic text using a preset instruction template, including: dividing the acquired structured numerical data into multiple dimensional indicators according to attributes, and normalizing them according to preset benchmark values ​​of each dimensional indicator; matching corresponding degree descriptive words from a preset financial dictionary enhancement unit according to the interval of the normalized values; extracting the prompt word skeleton from the instruction template, filling the names of the dimensional indicators and the matched degree descriptive words into the corresponding slots of the prompt word skeleton to generate semantic text fragments; concatenating the semantic text fragments with the corresponding indicator values, and adding preset role guidance prefixes to form descriptive semantic text.

3. The data fusion method based on natural language processing according to claim 1, characterized in that, The hybrid pre-trained model comprises a deep encoding layer, an autoregressive inference layer, an attention layer, an attention interaction layer, and a semantic calibration output layer. The deep encoding layer consists of multiple transformer encoding operators, used for fine-grained semantic modeling of unstructured text; it extracts named entities and financial terminology from the text and outputs local feature vectors. The autoregressive inference layer is based on a generative transformer architecture, used for logical chain inference of long texts; it parses implicit risk factors in the text and generates query vectors reflecting the global context. The attention layer has a low-rank adaptation matrix embedded in parallel. Without changing the model's pre-training parameters, it captures the non-linear coupling relationship between structured numerical indicators and unstructured text through low-order matrix operations, achieving feature alignment in the vertical domain. The attention interaction layer is a layer that uses a fusion attention mechanism to allocate weights. Calculate the cross-correlation degree between the query vector output by the autoregressive inference layer and the key-value vector output by the deep coding layer, and dynamically allocate feature weights; The semantic calibration output layer is used to perform continuous learning mechanisms and bias parameter calibration; The final output enhanced feature vector is dynamically adjusted based on the amount of contextual drift of the features over time.

4. The data fusion method based on natural language processing according to claim 3, characterized in that: The steps for achieving feature depth alignment within the attention layer include: maintaining the original pre-trained weight matrix of the attention layer in the hybrid pre-trained model in an untrainable state, and constructing an adaptation branch composed of low-order matrices next to the pre-trained weight matrix; inputting the modal features in parallel into the pre-trained weight matrix and the adaptation branch, and weighting and merging the low-order transformation features output by the adaptation branch with the original features output by the pre-trained weight matrix to obtain the corrected enhanced feature representation; and fine-tuning the low-order matrices in the adaptation branch to capture the correlation between numerical fluctuation patterns in descriptive semantic text and unstructured text semantics, so that features of different dimensions are mapped to the same high-dimensional vector space.

5. The data fusion method based on natural language processing according to claim 1, characterized in that, The steps for calculating the correlation between aligned feature vectors using a fusion attention mechanism include: performing a dot product operation on the query vector output by the generative architecture and the key vector output by the discriminative architecture; normalizing and scaling the dot product result; calculating the attention weight distribution of each feature using a hierarchical function; and using the attention weight distribution to perform a weighted summation on the value vector output by the discriminative architecture to obtain the correlation.

6. The data fusion method based on natural language processing according to claim 1, characterized in that, The steps of dynamically adjusting the feature weights between the query vector and the key vector using reinforcement learning include: taking the query vector, key vector, and the initially calculated attention weight distribution as the state input for reinforcement learning; adjusting the weights according to the current state; applying a nonlinear offset operator to the attention weight distribution to achieve dynamic fine-tuning of the contribution of global semantics and local fine-grained features; calculating the F1 score gain between the output of the hybrid pre-trained model and the preset label, feeding it back to reinforcement learning as a reward signal; and using the policy gradient algorithm to optimize the probability distribution of the adjustment action to obtain the optimal feature weights.

7. The data fusion method based on natural language processing according to claim 1, characterized in that, The steps for generating the fused target feature vector include: using the optimal feature weights dynamically adjusted by reinforcement learning, performing scalar multiplication operations on the fine-grained feature vector and the global semantic representation respectively to obtain the reconstructed local salient features and global semantic background features; concatenating the reconstructed local salient features and global semantic background features into vectors to construct an original fused feature space containing cross-modal interaction information; and performing high-dimensional projection mapping on the concatenated vector through a preset nonlinear activation function to generate the target feature vector by compressing redundant information and enhancing the feature response of key risk dimensions.

8. The data fusion method based on natural language processing according to claim 1, characterized in that, The continuous learning mechanism and the steps for enhancing feature output include: real-time collection of external macroeconomic data and industry sentiment information; extraction of a reference vector for the current time series using the hybrid pre-trained model; determination of context drift by measuring the distribution difference between the target feature vector and the reference vector on the time axis; construction of a calibration function based on the context drift; online micro-incremental adjustment of the bias operator of the output layer of the hybrid pre-trained model to compensate for semantic recognition deviations caused by changes in the macro environment; inputting the calibrated feature vector into a preset risk warning operator; combining the topological association strength in the knowledge graph for feature enhancement; and outputting enhanced features.

Images