A dynamic credibility quantification modeling method for cross-domain heterogeneous data

Abstract

The application relates to the technical field of data processing, in particular to a dynamic credibility quantification modeling method for cross-domain heterogeneous data, which comprises the following steps: through a data formatting and semantic analysis engine, cross-domain heterogeneous data is processed into standardized financial information elements; based on a set of dynamically updatable processing rules, the financial information elements are comprehensively evaluated from three dimensions of internal attributes, source credibility and cross-domain data correlation to generate multidimensional credibility; in a graph data environment, the multidimensional credibility is weighted, aggregated and time-limited adjusted in combination with context weight and a time decay function to obtain quantified credibility; and a hot loading mechanism is adopted to update core rules such as a source lookup table and weight configuration online and without interruption, so that the application can adapt to external environment changes in real time. The application realizes the fusion of cross-domain heterogeneous data through dynamic credibility quantification modeling.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a dynamic credibility quantitative modeling method for cross-domain heterogeneous data. BACKGROUND

[0002] With the deepening of data-driven decision-making, various industries are facing the explosive growth of massive data from different fields and different structures, i.e., cross-domain heterogeneous data. Effective fusion, in-depth analysis and intelligent application of these data are the core of promoting technological innovation, optimizing industrial decision-making and improving social governance capability.

[0003] However, these widely-sourced and variously-formatted data are of varying quality and reliability, and generally have problems such as data inconsistency, information conflict, outdated content, data missing and even false information. This lack of "data credibility" constitutes a core bottleneck in the data value chain. When making key decisions or training artificial intelligence models, if relying on "dirty data" of low credibility, not only the true value of the data cannot be tapped, but also incorrect analysis conclusions, biased algorithm models and significant economic losses or security risks may be caused. The traditional modeling method for cross-domain heterogeneous data has the limitations of being difficult to cope with the dynamic changes of data sources and data themselves, being difficult to establish a unified evaluation framework for heterogeneous data, and the existing credibility evaluation being limited to qualitative description.

[0004] Therefore, a dynamic credibility quantitative modeling method for cross-domain heterogeneous data is proposed. SUMMARY

[0005] The purpose of the present application is to provide a dynamic credibility quantitative modeling method for cross-domain heterogeneous data, which realizes the fusion of cross-domain heterogeneous data through dynamic credibility quantitative modeling.

[0006] To achieve the above purpose, the present application provides the following technical scheme:

[0007] A dynamic credibility quantitative modeling method for cross-domain heterogeneous data, comprising:

[0008] receiving and parsing cross-domain heterogeneous data, and converting the cross-domain heterogeneous data into financial information elements through a data formatting and semantic parsing engine;

[0009] processing the financial information elements based on a set of dynamically-updatable processing rules to generate multi-dimensional credibility, the processing rules including internal attribute value calculation, source lookup table matching and cross-domain data correlation degree statistics, and the multi-dimensional credibility including internal credibility, source credibility and correlation credibility;

[0010] The financial information element retrieves its adjacent node information in a graph data storage engine, weights and aggregates the multi-dimensional credibility of the adjacent nodes according to a preset context weight configuration file, adjusts the aggregation result by calling a time decay function, and generates a quantitative credibility;

[0011] The hot loading mechanism for establishing a data processing configuration based on processing rules receives an externally pushed configuration update package, performs an atomic replacement operation, and updates the source lookup table and the context weight configuration file online.

[0012] The cross-domain heterogeneous data includes structured data, semi-structured data, and unstructured data;

[0013] The structured data, semi-structured data, and unstructured data are subjected to key information extraction by a data formatting and semantic parsing engine; the structured data is directly extracted from a data table by a configured database adapter to obtain fields, values, and metadata, and the original association relationship is maintained to generate first key information; for semi-structured data including JSON and XML, key-value pairs in a nested structure are extracted by using an XPath path query language, and a hierarchical structure is parsed to generate second key information;

[0014] For unstructured data including text and reports, a natural language processing pipeline based on a pre-trained language model is used to sequentially perform named entity recognition and relationship extraction to generate third key information; and the first key information, the second key information, and the third key information are formatted by the data formatting and semantic parsing engine to generate the financial information element;

[0015] The specific process of generating the multi-dimensional credibility vector is as follows:

[0016] Based on a set of dynamically updateable processing rules, the financial information element is processed to generate a multi-dimensional credibility value, and the processing rules include internal attribute value calculation, source lookup table matching, and cross-domain data correlation degree statistics to generate the multi-dimensional credibility;

[0017] The multi-dimensional credibility includes internal credibility, source credibility, and correlation credibility.

[0018] The internal attribute value calculation generates the internal credibility, the source lookup table matching generates the source credibility, and the cross-domain data correlation degree statistics generates the correlation credibility.

[0019] The correlation credibility is specifically as follows:

[0020] The financial information element is represented by vectorization, the support relationship is identified by calculating the cosine similarity between vectors, and when the similarity score is higher than the preset positive threshold, a support edge with weight is established; entity relationship conflict detection is performed, and when the content of two information elements is contradictory and the conflict degree is higher than the preset negative threshold, a conflict edge with weight is established, forming a cross-domain information association graph;

[0021] Based on the cross-domain information association graph, a set of nodes with the highest source credibility is initialized as the starting point of trust, and a label propagation algorithm combining trust decay and conflict penalty mechanism is used, the credibility score of each node is accumulated according to the support strength of its neighbor nodes, and is reduced according to the strength of the conflict neighbor, and the iteration is performed until the scores of all nodes in the network converge, and the converged score is the association credibility.

[0022] The specific process of generating the quantitative credibility is:

[0023] In the cross-domain information association graph, the adjacent node information is retrieved according to the association relationship of the financial information element, the multi-dimensional credibility of the adjacent nodes is weighted and aggregated by a preset context weight configuration file, and a preliminary quantitative credibility is generated;

[0024] The weighted probability in the information propagation process is calculated by a random walk algorithm, the credibility of the node is updated, the stay threshold and the maximum number of steps are set, the weight algorithm is adjusted regularly, and the quantitative credibility is generated;

[0025] The time decay function is called to adjust the aggregation result.

[0026] The specific application mode of the time decay function is:

[0027] The time decay function adopts a nonlinear exponential decay mode to adjust the timeliness of the quantitative credibility of the financial information element, so that the quantitative credibility decreases with the increase of the interval between its generation time and the current time.

[0028] The hot loading mechanism of the data processing configuration is specifically:

[0029] The data processing configuration is a rule and parameter set for guiding the generation logic of the multi-dimensional credibility and the quantitative credibility;

[0030] A dynamic loading module of the data processing configuration is established, which is used to receive the configuration update package pushed by the outside in real time, and the configuration update package is a data carrier for transmitting new version configuration data;

[0031] When the configuration update package is received, an atomic replacement operation is performed, and the existing source lookup table and context weight configuration file are called in real time.

[0032] The data processing configuration comprises at least a source lookup table and a context weight configuration file.

[0033] Compared with the prior art, the present application has the following advantages:

[0034] 1. The present application significantly improves the accuracy and reliability of the evaluation results by designing a comprehensive evaluation system for credibility quantification from multiple dimensions and multiple levels. This method can not only uniformly process structured, semi-structured and unstructured heterogeneous data, but also creatively combines the internal quality (integrity, standardization) of information, external credibility (source historical score) and horizontal correlation (cross-domain correlation verification degree). In particular, by constructing a cross-domain information correlation graph for conflict and support analysis, the evaluation is no longer isolated, but is comprehensively considered in an information network, thereby revealing the true credibility of information more deeply and comprehensively.

[0035] 2. The present application realizes the high automation and intelligence of the credibility quantification modeling process by introducing various technologies such as graph calculation, label propagation algorithm and time decay function. This method can automatically construct a complex relationship graph between information elements, simulate the propagation, conflict and decay process of trust in the network, and objectively calculate the credibility of each information element in a specific context environment and time, thereby improving the automation level and depth of credibility evaluation, reducing the dependence on manual configuration of complex rules, and enabling the evaluation results to dynamically reflect the influence of information timeliness, making the entire modeling process more intelligent and efficient.

[0036] 3. The present application solves the limitations of the existing technology that the evaluation model is fixed and cannot adapt to the dynamic changing data environment and application requirements by designing a context-aware dynamic weight fusion model and a closed-loop feedback iteration mechanism based on reinforcement learning. This method not only intelligently adjusts the importance weight of each evaluation dimension according to specific application scenarios such as financial early warning and scientific research review, but also automatically updates the reputation rating of data sources according to the good or bad evaluation results through continuous learning, realizing the self-evolution and adaptive optimization of the model. This makes the entire evaluation system highly intelligent and automated, can continuously maintain the accuracy and timeliness of the evaluation in the long-term operation, and significantly reduces the cost of manual intervention and model maintenance. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 A method flowchart of a dynamic credibility quantification modeling method for cross-domain heterogeneous data;

[0038] Figure 2 An operation sequence diagram of a dynamic credibility quantification modeling method for cross-domain heterogeneous data. DETAILED DESCRIPTION

[0039] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] Example 1:

[0041] Please see Figure 1 and Figure 2 This invention provides a dynamic and reliable quantitative modeling method for cross-domain heterogeneous data, the technical solution of which is as follows:

[0042] A dynamic and reliable quantitative modeling method for cross-domain heterogeneous data includes:

[0043] Receive and parse cross-domain heterogeneous data, and transform the cross-domain heterogeneous data into financial information elements through a data formatting and semantic parsing engine;

[0044] Based on a set of dynamically updatable processing rules, the financial information elements are processed to generate a multidimensional credibility vector. The processing rules include calculating intrinsic attribute values, matching source lookup tables, and statistical analysis of cross-domain data correlation to generate the multidimensional credibility vector.

[0045] Based on financial information elements, the system retrieves the information of adjacent nodes in the graph data storage engine. According to the preset context weight configuration file, the multidimensional credibility vectors of the retrieved adjacent nodes are weighted and aggregated. The time decay function is called to adjust the aggregation result and generate quantified credibility.

[0046] A hot-loading mechanism for data processing configuration is established based on processing rules. It receives configuration update packages pushed from external sources, performs atomic replacement operations, and updates the source lookup table and context weight configuration file online.

[0047] The cross-domain heterogeneous data includes structured data, semi-structured data, and unstructured data;

[0048] By integrating data formatting and semantic parsing engines, the first, second, and third key information are extracted, formatted, and financial information elements are generated.

[0049] The first key information is that the structured data is connected to the official trading data platform through a preset database adapter; fields and values, such as company stock codes, real-time transaction prices, price changes, and trading volumes, are directly extracted from this connection; at the same time, the data's metadata, such as the data source institution and timestamp, is also obtained.

[0050] The second key information is for semi-structured data, including JSON and XML, which uses XPath path query language to extract key key-value pairs in nested structures and parse the hierarchical structure as contextual metadata; for example, using XPath path query language to extract key information from complex nested JSON structures, parsing out key-value pairs such as announcement titles, release dates, and "technical codes" or "key performance indicators" in technical details, and understanding the hierarchical relationship between these information to generate the second key information;

[0051] The third key information is for unstructured data, including text and reports, based on a pre-trained language model natural language processing pipeline, which sequentially performs named entity recognition and relation extraction, such as finding key entities in the text such as "a certain company" and "the company's CEO"; then relation extraction is performed to understand that the core event is "a certain company releases a certain technology", and the third key information is generated.

[0052] By designing specialized and efficient extraction paths for structured, semi-structured, and unstructured data, and integrating and formatting the extracted multi-source and heterogeneous key information through a unified engine, the problem of information silos and parsing difficulties caused by diverse data sources and inconsistent formats in the financial field is solved. The resulting financial information elements achieve data standardization and structuring, integrating objective data from the trading market and core semantic relationships from text reports, providing a unified, complete, and semantically deep data foundation for subsequent accurate and comprehensive credible quantitative analysis.

[0053] Based on financial information elements, a multidimensional credibility value is generated through processing rules. The specific process is as follows:

[0054] Based on a set of dynamically updatable processing rules, the financial information elements are processed to generate multidimensional credibility values. The processing rules include calculation of intrinsic attribute values, matching of source lookup tables, and statistics of cross-domain data correlation to generate the multidimensional credibility values.

[0055] The multidimensional credibility vector includes intrinsic credibility, source credibility, and association credibility;

[0056] The intrinsic attribute values ​​in the processing rules are calculated based on a preset quality rule base. This automatically verifies the data integrity, format standardization, and numerical logic of the financial information element. Based on preset deduction standards, points are accumulated for rule violations to generate an intrinsic credibility value. The initial intrinsic credibility value of a financial information element is 1.0. The quality rule base stipulates that "stock codes must be 6 digits," while the stock code field of this information element is "A00123," violating the format standardization rule. The preset penalty for this rule is 0.2. Simultaneously, the rule base also stipulates that "core events cannot be empty," and this information element complies with this rule. Therefore, the final intrinsic credibility value of the financial information element is calculated as 1.0 - 0.2 = 0.8.

[0057] The source lookup table matching in the processing rule is to perform a query and match in the source lookup table based on the source identifier and domain affiliation of the financial information element, and directly obtain the preset reputation score corresponding to the source and domain. The reputation score is the source credibility.

[0058] The cross-domain data correlation statistics in the processing rules are as follows: the financial information elements are used as nodes to construct a cross-domain information correlation graph. Based on the consistency or contradiction between the financial information elements, weighted support and conflict edges are established. Through graph computing algorithms, the support weights received by each node are accumulated and the conflict weights received are subtracted. Finally, the correlation credibility is generated after processing by a normalization function.

[0059] By combining three core dimensions—the intrinsic quality verification of information itself (intrinsic credibility calculation), the authority assessment of information sources (source lookup table matching), and the consistency verification of information in the external environment (cross-domain data correlation statistics)—a three-dimensional, multi-level credibility assessment system is constructed. This system overcomes the one-sidedness and uncertainty brought about by traditional single assessment dimensions. The resulting multi-dimensional credibility value can more comprehensively, accurately, and objectively reflect the overall credibility level of a financial information element.

[0060] The specific process for calculating the association credibility is as follows:

[0061] The financial information elements are vectorized using a pre-trained sentence vector model. This pre-trained sentence vector model maps the core events and semantic information contained in the financial information elements into high-dimensional mathematical vectors through a multi-layer neural network. The multi-layer neural network includes an embedding layer, an encoding layer, a context understanding layer, a pooling layer, and an output layer. Through this multi-layer neural network structure, financial information elements can be accurately converted into high-dimensional vectors, more precisely capturing the key semantics and sentiment in events. The embedding and encoding layers enhance the expressive power of information, and the context understanding layer enhances the model's understanding of complex financial contexts, thereby optimizing the accuracy of market prediction and risk assessment.

[0062] After vectorization, the cosine similarity between any two information element vectors is calculated to automatically identify the support relationship. If the calculated similarity score is higher than a preset positive threshold (e.g., 0.9), it is determined that the two information elements are highly consistent in content and corroborate each other. Thus, a weighted support edge is established between the nodes representing them, and the weight of the edge is based on the similarity score.

[0063] The system performs entity relationship conflict detection to identify financial information elements that contradict each other in content. Entity relationship conflict detection involves detecting information about the same entity and attribute but with significantly different descriptions. For example, the core relationship of financial information element A is (Company A, revenue growth rate, +15%), while financial information element B is (Company B, revenue growth rate, -5%), indicating a clear numerical conflict. When the degree of conflict exceeds a preset negative threshold, a weighted conflict edge is established between nodes A and B, with the weight representing the degree of conflict. A cross-domain information association graph is formed based on the entity relationships of the financial information elements, the weighted support edge, and the weighted conflict edge.

[0064] The label propagation algorithm iterates based on a cross-domain information association graph. The label propagation algorithm requires parameter setting and network initialization. Specifically, a trust decay coefficient is preset, for example, 0.85, and a convergence threshold is used to determine whether the iteration terminates, for example, 0.0001. During initialization, the label propagation algorithm selects the group of financial information elements with the highest source credibility value in the cross-domain information association graph, defines it as the starting point of trust, and sets its initial credibility score to 1.0. The initial credibility scores of other financial information elements in the cross-domain information association graph are uniformly set to a neutral value, for example, 0.5.

[0065] After initialization, the label propagation algorithm begins iterative calculation round by round. In each iteration, each financial information element node in the graph updates its score based on the state of its neighboring nodes in the previous round. The specific update logic of a node is to calculate the strength of all its supporting neighbors, multiply the current score of each supporting neighbor node by the corresponding supporting edge weight, and then accumulate them. The strength of all its conflicting neighbors is calculated in the same way. According to the trust decay and conflict penalty mechanism, the accumulated support strength is multiplied by the trust decay coefficient and then subtracted from the accumulated conflict strength to obtain an update amount. The node's new score in this round is the result of adding its score in the previous round to this update amount. The result is constrained to the range of 0 to 1 by a normalization function.

[0066] The algorithm continues to iterate and update until the scores of each node in the network converge. At the end of each iteration, the changes in the scores of all nodes in the network are checked. When the sum is less than the preset convergence threshold, the network is considered to have converged, and the iteration process will automatically terminate. Once the algorithm converges, the final stable score of each financial information element node is the correlation credibility.

[0067] Based on the cross-domain information association graph, a set of nodes with the highest source credibility is initialized as the starting point of trust. The label propagation algorithm, which combines trust decay and conflict penalty mechanisms, accumulates the credibility score of each node according to the support strength of its neighboring nodes and reduces it according to the strength of conflicting neighbors. After iteration, the scores of each node in the network converge, and the converged score is the association credibility.

[0068] By combining cosine similarity calculation of semantic vectors with entity relationship conflict detection, an objective and automated method for constructing information association graphs is realized, which can accurately identify deep semantic support and core logical conflicts between information. The label propagation algorithm adopted is initialized by setting the starting point of trust and incorporates trust decay and conflict penalty mechanisms, which accurately simulates the propagation, enhancement and weakening of trust in information networks in the real world.

[0069] The quantitative credibility is generated based on the cross-domain information association graph. The specific process is as follows: First, all adjacent nodes directly connected by supporting or conflicting edges are retrieved for the target financial information element. Then, a preset context weight configuration file is loaded. The context weight configuration file is a rule base that can be dynamically updated. It defines the basic importance weights that adjacent nodes of different types or sources should have in a specific analysis scenario (e.g., when analyzing company financial reports or market news). Using the basic weights in the configuration file, combined with the existing edge weights (representing the tightness of content association) in the cross-domain information association graph, the multidimensional credibility of each adjacent node is weighted and aggregated to generate a preliminary quantitative credibility.

[0070] The initial quantification confidence reflects the relationship between the target information element and its direct neighbors. In order to obtain a more global and robust final result, the initial quantification confidence is updated by a random walk algorithm. The random walk algorithm evaluates the true influence of a node by simulating the propagation process of information in the entire cross-domain information association graph. Starting from the target node, the algorithm walks in the cross-domain information association graph with weighted probability. It is more likely to jump to neighboring nodes connected by high-weight support edges, and almost never jumps to nodes with conflicting relationships.

[0071] The random walk algorithm iterates multiple times until the confidence scores of all nodes stabilize. This stabilization is controlled by two preset parameters: a dwell threshold and a maximum number of steps. The dwell threshold is used to determine convergence when the total change in scores between two consecutive iterations is less than this threshold. This dwell threshold also serves as a safety mechanism to prevent infinite loops. The final stable score of the target node is the quantized confidence score, ensuring that the final quantized confidence score is a stable solution obtained after sufficient iteration and network game theory, rather than an arbitrary intermediate snapshot. This improves the mathematical rigor and objective reliability of the evaluation results. By introducing a maximum number of steps as a mandatory termination condition, a reliable safety mechanism is provided, effectively avoiding wasted computational resources or system crashes due to non-convergence in extreme cases, ensuring the efficiency and robustness of the entire quantization process.

[0072] A two-stage quantification method combining "local aggregation" and "global correction" is adopted. The first stage is weighted aggregation, which introduces a dynamically updatable context weight configuration file to ensure that the initial credibility assessment can accurately reflect the focus of the specific analysis scenario, giving it high scenario adaptability. The second stage is a random walk algorithm that simulates the propagation of information in the entire network, which globally verifies and corrects the initial results, resulting in a more objective and reliable assessment conclusion that is both targeted and globally robust.

[0073] After the quantitative credibility of financial information elements has undergone the previous steps, the quantitative credibility is adjusted for timeliness to ensure that the final output quantitative credibility can truly reflect the value loss of information that occurs naturally over time. Specifically, the quantitative credibility of financial information elements is adjusted for timeliness by using a non-linear exponential decay mode through a time decay function, so that the quantitative credibility decreases as the time interval between its generation time and the current time increases.

[0074] First, for each financial information element (such as news, reports, announcements, etc.), an initial credibility value is assessed and assigned based on factors such as its source, content reliability, and historical records. This initial value represents the credibility of the information at the time of its publication. The time difference between the publication time of each financial information element and the current time is calculated; for example, if a financial information element was published several months ago and the current time is today, the time interval is calculated based on the difference between these two times.

[0075] The initial credibility of financial information elements is adjusted using a non-linear exponential decay model. As time intervals increase, the credibility of the information gradually decreases. This means that the earlier the information is released, the more pronounced the credibility decay, because the timeliness of the information becomes less important over time, and the reliability of the information may also be affected.

[0076] The rate at which credibility decays is controlled by a parameter called the decay factor. This parameter determines how quickly information decays; a larger decay factor causes the credibility of information to decline faster, while a smaller decay factor means that the information can maintain its credibility for a longer period. This example relates to the financial field, and the decay factor is set to 0.05 because the influence of such information typically declines significantly in a short period. The decay factor is not fixed; it can be dynamically adjusted with changes in time, market conditions, or data quality. For example, during periods of market volatility, the timeliness of certain information may become more critical, and in such cases, the value of the decay factor can be appropriately increased.

[0077] Through the above steps, combined with the initial credibility and decay calculation of each financial information element, the timeliness-adjusted quantitative credibility of each piece of information is finally obtained. The quantitative credibility reflects the value loss that information naturally generates over time, ensuring that the final output credibility value can truly reflect the timeliness of financial information.

[0078] By introducing a nonlinear exponential decay model to adjust the timeliness of the credibility of financial information elements, the value loss of information over time can be more accurately reflected. This method ensures that financial information can automatically adapt to changes in the timeliness of information during processing, so that outdated information will not have an excessive impact on decision-making. At the same time, by dynamically adjusting the decay factor, it is possible to flexibly respond to changes in the market environment, improve the adaptability and real-time performance of the model, and help improve the accuracy and reliability of financial analysis and risk assessment.

[0079] The hot-loading mechanism of the data processing configuration is as follows:

[0080] The data processing configuration is a set of externally modifiable rules and parameters that guide the calculation logic for generating multidimensional credibility and quantitative credibility. It includes at least the source lookup table and the context weight configuration file. The content of this configuration directly determines the credibility score finally calculated for each financial information element.

[0081] Establish a dynamic loading module for data processing configuration to receive configuration update packages pushed from the outside in real time. The configuration update package is a data carrier for transmitting new version configuration data.

[0082] Upon receiving the configuration update package, an atomic replacement operation is performed to seamlessly and instantly replace the current configuration file (i.e., the currently effective source lookup table and context weight configuration file) that is being called in real time in memory with the new version configuration file contained in the configuration update package, so as to ensure that the data processing flow is not interrupted.

[0083] The updated configuration is synchronized to the processing flow. The final effect is that from the moment of update, all subsequent financial information elements will automatically adopt the new rules and new data in the updated source lookup table and context weight configuration file when performing credibility calculations. The hot loading mechanism supports multi-threaded concurrent execution and synchronizes the updated configuration.

[0084] By employing a hot-loading mechanism, the "data processing configuration" (such as source reputation and context weight) that determines the credibility calculation criteria is completely separated from the core program logic of the system. This design allows adjustments and optimizations to the evaluation rules to be completed without modifying the code or restarting the service, greatly improving the flexibility and maintainability of the entire system. More importantly, through atomic replacement operations, configuration updates can be performed in real time, seamlessly, and without interrupting the core data processing flow.

[0085] This embodiment provides an end-to-end, dynamic financial information credibility quantification solution. First, it solves the problem of unified processing of heterogeneous data across domains through a powerful parsing engine, which can transform data from diverse sources and with disordered formats into standardized financial information elements. On this basis, a multi-dimensional evaluation and graph weighted aggregation method combines multiple factors such as intrinsic information quality, source reputation, contextual relevance, and time decay, making the final quantitative credibility assessment extremely comprehensive, accurate, and close to the complexity of the real world. By introducing a hot-loading mechanism, the entire evaluation logic can be updated online without downtime, ensuring that the system can operate with high availability 24 / 7 while continuously and quickly adapting to the ever-changing financial information environment.

[0086] Example 2:

[0087] This embodiment uses a specific financial information element as an example to demonstrate the entire process from receiving it to the final quantitative credibility output.

[0088] First, an unstructured data message was received from a "financial blog". The message was about "Company A's stock price is expected to rise due to a new technological breakthrough". The message was published one day ago. The data formatting and semantic parsing engine was activated and the article was parsed through the natural language processing pipeline. It was successfully transformed into a structured financial information element, which included key information such as source, core entity (Company A), event (expected stock price increase) and timestamp.

[0089] Next, a multidimensional credibility score is generated for the financial information element based on the processing rules. During the calculation of intrinsic attribute values, the quality rule base detected a minor formatting error in the company stock code cited in the article, therefore its intrinsic credibility score is calculated to be 0.9 (out of 1.0, minus 0.1). Subsequently, through source lookup table matching, the historical reputation score of "a certain financial blog" is 0.7, which is the source credibility score. Finally, in the constructed cross-domain information association graph, the financial information element receives support from two high-weight nodes and a weak conflict from a low-weight node. After convergence via the label propagation algorithm, its association credibility score is calculated to be 0.85, and the multidimensional credibility vector of this information element is generated as [0.9, 0.7, 0.85].

[0090] Based on this multidimensional credibility, weighted aggregation is performed in the graph to generate a preliminary quantitative credibility. Under the guidance of the context weight configuration file of the "market news" analysis scenario, "official announcements" in the neighboring nodes of this information element are given high weight, and other "blog" type neighbors are given low weight. After weighted aggregation calculation, the preliminary quantitative credibility obtained after combining the influence of each neighbor node is 0.76.

[0091] The initial credibility was globally corrected and its timeliness adjusted. By initiating a random walk algorithm, and considering the strong correlation between the neighboring nodes of this information element and other highly credible official information nodes in the network, the credibility score was corrected from 0.76 to 0.82 after the algorithm converged iteratively. The time decay function was then invoked to fine-tune its credibility based on its high timeliness sensitivity coefficient in the "market news" domain and the one-day time interval since its publication. Ultimately, the final quantitative credibility of this financial information element after timeliness adjustment was 0.80.

[0092] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A dynamic trustworthiness quantification modeling method for cross-domain heterogeneous data, characterized in that, The application relates to a method for processing cross-domain heterogeneous data, and belongs to the technical field of data processing. The method comprises the following steps: receiving and analyzing cross-domain heterogeneous data, and converting the cross-domain heterogeneous data into financial information elements through a data formatting and semantic analysis engine; the cross-domain heterogeneous data comprises structured data, semi-structured data and unstructured data; key information of the structured data, the semi-structured data and the unstructured data is extracted through the data formatting and semantic analysis engine; the structured data is directly extracted from a data table through a configured database adapter, and fields, values and metadata of the structured data are kept in the original associated relationship to generate first key information; for the semi-structured data, including JSON and XML, key-value pairs in a nested structure are extracted through an XPath path query language, and hierarchical structures are analyzed to generate second key information; for the unstructured data, including text and reports, a natural language processing pipeline based on a pre-trained language model is used to sequentially perform named entity recognition and relation extraction to generate third key information; and the first key information, the second key information and the third key information are formatted through the data formatting and semantic analysis engine to generate the financial information elements; a set of dynamically updateable processing rules are used to process the financial information elements to generate multi-dimensional credibility, the processing rules comprise internal attribute value calculation, source lookup table matching and cross-domain data correlation degree statistics, and the multi-dimensional credibility comprises internal credibility, source credibility and correlation credibility; based on the financial information elements, adjacent node information is retrieved in a graph data storage engine, the multi-dimensional credibility of the retrieved adjacent nodes is weighted and aggregated according to a preset multi-dimensional credibility context weight configuration file, a time decay function is used to adjust the aggregation result to generate a quantitative credibility; 2. The dynamic trustworthiness quantification modeling method of cross-domain heterogeneous data according to claim 1, characterized in that, a hot loading mechanism of data processing configuration is established based on the processing rules to receive an externally pushed configuration update package, perform an atomic replacement operation, and update the source lookup table and the context weight configuration file online. The specific process of generating the multi-dimensional credibility vector is as follows: a set of dynamically updateable processing rules are used to process the financial information elements to generate multi-dimensional credibility values, the processing rules comprise internal attribute value calculation, source lookup table matching and cross-domain data correlation degree statistics, and the multi-dimensional credibility comprises internal credibility, source credibility and correlation credibility; the internal attribute value calculation generates the internal credibility, the source lookup table matching generates the source credibility, and the cross-domain data correlation degree statistics generates the correlation credibility.

3. The method of claim 1, wherein, The correlation credibility is specifically as follows: the financial information elements are represented in a vector form, cosine similarity between vectors is calculated to identify support relationships, when a similarity score is higher than a preset positive threshold, a support edge with a weight is established; entity relationship conflict detection is performed, when two information element contents are contradictory and a conflict degree is higher than a preset negative threshold, a conflict edge with a weight is established, and a cross-domain information correlation graph is formed. Based on the cross-domain information association graph, a set of nodes with the highest source credibility is initialized as the starting point of trust, and a label propagation algorithm combined with trust decay and conflict penalty mechanism is used. The credibility score of each node is accumulated according to the support strength of its neighbor nodes and is reduced according to the strength of the conflict neighbors. Through iteration, the scores of the nodes in the network converge, and the converged scores are the association credibility.

4. The dynamic trustworthiness quantification modeling method of cross-domain heterogeneous data according to claim 1, characterized in that, The specific process of generating the quantitative credibility is: In the cross-domain information association graph, the adjacent node information is retrieved according to the association relationship of the financial information element, the multi-dimensional credibility context weight configuration file is used to weight and aggregate the multi-dimensional credibility of the adjacent nodes, and the preliminary quantitative credibility is generated; The weighted probability in the information propagation process is calculated by the random walk algorithm, the credibility of the node is updated, the stay threshold and the maximum number of steps are set, the weight algorithm is adjusted regularly, and the quantitative credibility is generated; The time decay function is called to adjust the aggregation result.

5. The dynamic trustworthiness quantification modeling method of cross-domain heterogeneous data according to claim 4, characterized in that, The specific application mode of the time decay function is: The time decay function uses a nonlinear exponential decay mode to adjust the timeliness of the quantitative credibility of the financial information element, so that the quantitative credibility decreases as the interval between its generation time and the current time increases.

6. The dynamic trustworthiness quantification modeling method of cross-domain heterogeneous data according to claim 1, characterized in that, The hot loading mechanism of the data processing configuration is specifically: The data processing configuration is a set of rules and parameters that guide the generation logic of the multi-dimensional credibility and the quantitative credibility; A dynamic loading module of the data processing configuration is established to receive the configuration update package pushed by the external in real time. The configuration update package is a data carrier for transmitting new version configuration data; When the configuration update package is received, an atomic replacement operation is performed, and the existing source lookup table and context weight configuration file are called in real time.

7. The dynamic trustworthiness quantification modeling method of cross-domain heterogeneous data according to claim 6, characterized in that, The data processing configuration at least includes a source lookup table and a context weight configuration file.

Images