Market scale dynamic evaluation model construction method based on multi-source heterogeneous data fusion

Abstract

The invention discloses a market scale dynamic evaluation model construction method based on multi-source heterogeneous data fusion, and relates to the field of market scale dynamic evaluation, and the method comprises the steps: dividing multi-source heterogeneous data into a plurality of data mode sets based on data modes; generating at least one data fusion and feature construction scheme based on the core market index and all the data modal sets; according to the generated data fusion and feature construction schemes, calculating a scheme feature stability index under each data fusion and feature construction scheme; constructing a market scale dynamic evaluation model, and outputting a core quantitative index prediction value; and based on the optimal data fusion and feature construction scheme and the market scale dynamic evaluation model, obtaining a market scale dynamic evaluation report. The method has the advantages that a multi-source data fusion scheme is automatically optimized and solidified, the time sequence consistency of input features is ensured, and the robustness, repeatability and interpretability of a market scale dynamic evaluation result are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of dynamic market size assessment models, specifically to a method for constructing a dynamic market size assessment model based on the fusion of multi-source heterogeneous data. Background Technology

[0002] In the digital economy era, dynamic and accurate assessment of market size is the cornerstone of business decision-making, investment analysis, and policy formulation. With the development of information technology, the data available to assessors has expanded from single structured reports to a massive, multi-source, and heterogeneous "big data" ecosystem. This mainly includes three core data modalities: First, time-series numerical sequences, such as monthly sales figures, product price indices, and infrastructure stock, which directly reflect the quantitative trends of the market with regular timestamps and values; Second, unstructured text, such as industry policy documents, social media sentiment, and patent technology documents, which contain qualitative driving forces such as policy orientation, consumer sentiment, and technological breakthroughs that influence the market; Third, relational graph data, such as supply chain networks, corporate equity investment relationships, and strategic alliances, which reveal the connections and influences among market participants through network structures. Theoretically, integrating this multi-dimensional information can build an unprecedented, comprehensive, and profound market insight system. Therefore, how to systematically integrate heterogeneous data such as time-series, text, and graphs, and extract stable and reliable features to drive assessment models, has become a cutting-edge topic and urgent need in the field of big data analysis and intelligent decision-making.

[0003] The current assessment of market size relies heavily on core quantitative indicators derived from fixed calculation formulas (such as market size = cumulative sales volume × average product price). This approach is essentially a post-hoc statistical description of completed transactions. Its fundamental flaw lies in the fact that the arithmetic expressions upon which it is based can only mechanically reflect numerical results of "what," but are completely powerless to explain the deep-seated driving logic of "why" and "how" changes will occur. Market changes are not directly determined by a few isolated figures such as sales volume and average price, but rather by a dynamic process driven by a complex mix of multi-source, heterogeneous factors, including policy guidance (unstructured text), market sentiment (emotional polarity), supply chain relationships (relationship graphs), and technological trends. These key pieces of information that determine market trends, due to their unstructured and semantic nature, are completely shielded from the assessment system by traditional quantitative formulas. This results in current technology only presenting a static, superficial, and uninterpretable numerical snapshot, unable to provide in-depth quantification, attribution analysis, or forward-looking insights into the market state. Summary of the Invention

[0004] To address the aforementioned technical issues, this paper presents a method for constructing a dynamic market size assessment model based on the fusion of multi-source heterogeneous data. This technical solution resolves the problem mentioned in the background that the lack of dynamic analysis of complex factors such as policy guidance, market sentiment, supply chain relationships, and technological trends has resulted in existing technologies only presenting a static, superficial, and uninterpretable digital snapshot, failing to provide in-depth quantification, attribution analysis, and forward-looking insights into the market status.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A method for constructing a dynamic market size assessment model based on multi-source heterogeneous data fusion includes:

[0007] Acquire multi-source heterogeneous data of the target evaluation market, and divide the multi-source heterogeneous data into multiple data modality sets based on data modality;

[0008] Based on core market metrics and all data modal sets, generate at least one data fusion and feature construction scheme;

[0009] Based on the generated data fusion and feature construction schemes, calculate the scheme feature stability index for each data fusion and feature construction scheme.

[0010] Based on the stability index of the scheme characteristics, a dynamic market size assessment model is constructed based on a multi-task regression neural network architecture, and the predicted values ​​of core quantitative indicators are output.

[0011] Based on the optimal data fusion and feature construction scheme and the dynamic market size assessment model, a dynamic market size assessment report is obtained.

[0012] Preferably, the step of generating at least one data fusion and feature construction scheme based on core market indicators and all data modal sets specifically includes:

[0013] The core market indicators for market size are set, and the core market indicators include at least: market size, annual incremental size, market share and market growth rate.

[0014] Based on the data structure and semantic expression, a pre-defined feature engineering strategy library is constructed.

[0015] Based on each core market metric, at least one feature extraction strategy is determined for each data modality set from a pre-defined feature engineering strategy library;

[0016] The combination of all feature extraction strategies corresponding to each data modality set is used as the set of candidate feature strategies for that modality.

[0017] Based on the historical information entropy of each data modality set and the Pearson correlation coefficient of the core market indicators, the initial modality fusion weights are obtained through a weighted synthesis formula.

[0018] Select a feature extraction strategy from the set of candidate feature strategies corresponding to each set of data modalities, and assign an initial modality fusion weight to combine them into a data fusion and feature construction scheme;

[0019] By iterating through all data modal sets and combining the feature extraction strategies with the initial fusion weights, several data fusion and feature construction schemes are obtained.

[0020] Preferably, the step of calculating the feature stability index under each data fusion and feature construction scheme according to the generated data fusion and feature construction scheme specifically includes:

[0021] For each generated data fusion and feature construction scheme, the historical data within the preset historical evaluation window is divided into several fixed time windows. Modal features of each fixed time window are extracted through the preset feature extraction strategy in the scheme.

[0022] Based on the initial fusion weights, local determinism scores, and modal characteristics of each scheme within a fixed time window, the contribution of each modal information within that fixed time window is calculated.

[0023] For each fixed time window, the information contribution of all modalities within that window is integrated into the feature vector of the fusion scheme for that window;

[0024] Calculate the coefficient of variation of the feature vectors corresponding to all fixed time windows to form a coefficient of variation sequence;

[0025] The coefficient of variation refers to the ratio of the standard deviation of the eigenvector to the mean. The smaller the ratio, the more balanced the contribution of each mode is distributed and the smaller the fluctuation in the time slice.

[0026] The average value of the coefficient of variation sequence is the characteristic stability index of the scheme.

[0027] Preferably, the step of constructing a dynamic market size assessment model based on a multi-task regression neural network architecture, according to the stability index of the scheme characteristics, and outputting predicted values ​​of core quantitative indicators specifically includes:

[0028] Based on the stability index of the scheme features, the scheme with the smallest index value is selected as the optimal data fusion and feature construction scheme, and the modality fusion weight and feature extraction strategy corresponding to the scheme are extracted.

[0029] A neural network architecture with multi-task regression is adopted, with the feature vector of the fusion scheme generated by the optimal solution as the model input and the predicted values ​​of four core quantitative indicators as the model output.

[0030] The neural network architecture for the multi-task regression consists of a shared low-level feature encoding network and four independent regression output heads;

[0031] Collect and record historical market size data, and obtain the historical fusion scheme feature vectors according to the feature vector extraction method of the fusion scheme in the optimal data fusion and feature construction scheme.

[0032] Obtain actual market size survey data for the same time window, and calculate the true value of the core quantitative indicator corresponding to that time window based on the quantitative formula of the indicator.

[0033] Based on the obtained historical fusion scheme feature vectors and the true values ​​of core quantitative indicators, a training sample set is constructed and divided into a training set, a validation set, and a test set in an 8:1:1 ratio.

[0034] Preprocessing of the training sample set includes at least: data normalization, outlier removal, and interpolation;

[0035] Based on the mean square error formula, the sum of squared deviations between the predicted and actual values ​​of each core quantitative indicator is quantified, and a loss function is constructed.

[0036] With the goal of minimizing the loss function, the Adam optimizer is used to train the model using the training set data, and an early stopping mechanism is introduced to prevent overfitting.

[0037] After training, its parameters are fixed to obtain a deployable dynamic market size assessment model, and the model is used to output predicted values ​​of core quantitative indicators.

[0038] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0039] This invention proposes a method for constructing a dynamic market size assessment model based on multi-source heterogeneous data fusion. By introducing a feature stability index as the core evaluation criterion for comprehensively selecting different data fusion and feature construction schemes, and calculating the dispersion of the feature vector of the fusion scheme composed of the contribution of multimodal information under each scheme within each time window based on historical backtesting windows, the average value of this coefficient of variation sequence is used as the feature stability index. This enables automated and objective optimization and screening of numerous candidate fusion schemes, ensuring that the feature representation constructed by the finally selected optimal scheme remains highly consistent across different historical time points. This avoids the instability of the subsequent training evaluation model output due to drastic fluctuations in the features themselves, greatly improving the robustness, repeatability, and interpretability of the dynamic market size assessment results, and providing a reliable and consistent quantitative benchmark for data-driven market decision-making. Attached Figure Description

[0040] Figure 1 This is a flowchart of a method for constructing a dynamic market size assessment model based on multi-source heterogeneous data fusion according to the present invention.

[0041] Figure 2 The present invention generates at least one data fusion and feature construction scheme flowchart based on core market indicators and all data modal sets;

[0042] Figure 3 The flowchart of the present invention calculates the feature stability index under each data fusion and feature construction scheme according to the generated data fusion and feature construction scheme;

[0043] Figure 4 This is a structural diagram of the electronic device proposed in this invention;

[0044] Figure 5 This is a schematic diagram of the structure of the computer-readable storage medium proposed in this invention. Detailed Implementation

[0045] The following description is intended to disclose the invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art.

[0046] Reference Figure 1 As shown, a method for constructing a dynamic market size assessment model based on multi-source heterogeneous data fusion includes:

[0047] Acquire multi-source heterogeneous data of the target evaluation market, and divide the multi-source heterogeneous data into multiple data modality sets based on data modality;

[0048] Based on core market metrics and all data modal sets, generate at least one data fusion and feature construction scheme;

[0049] Based on the generated data fusion and feature construction schemes, calculate the scheme feature stability index for each data fusion and feature construction scheme.

[0050] Based on the stability index of the scheme characteristics, a dynamic market size assessment model is constructed based on a multi-task regression neural network architecture, and the predicted values ​​of core quantitative indicators are output.

[0051] Based on the optimal data fusion and feature construction scheme and the dynamic market size assessment model, a dynamic market size assessment report is obtained.

[0052] To achieve a dynamic, robust, and interpretable assessment of market size based on multi-source heterogeneous data, the first step is to address the issues of data structuring and processing. Therefore, the first step, based on the inherent structure and semantics of the data, divides the multi-source heterogeneous data into homogeneous data modal sets according to time-series numerical sequences, unstructured text, and relational graphs. This lays the foundation for subsequent targeted feature engineering, avoiding information loss or noise amplification caused by a "one-size-fits-all" approach. After clarifying the data foundation, the core issue of "how to fuse" needs to be addressed. Therefore, the second step, based on core market indicators, matches feature extraction strategies to each modality from a pre-defined feature engineering strategy library. Initial fusion weights are generated by combining information entropy and relevance weights. A large number of candidate data fusion and feature construction schemes are automatically constructed through iterative combinations, transforming subjective design relying on expert experience into a quantifiable search optimization problem. The most reliable scheme is selected from numerous candidates. The third step introduces a feature stability index. By calculating the average value of the coefficient of variation sequence of the fused feature vectors over a historical time window, the intrinsic consistency of the feature representation constructed by the evaluation scheme in the time dimension is quantified, thereby selecting the scheme that provides the best input to the evaluation model. The stable optimal solution breaks through the limitations of traditional fusion methods that only focus on instantaneous correlation and ignore time-series robustness. After determining the optimal data fusion rules, it is necessary to construct an accurate mapping from fusion features to specific market indicators. Therefore, the fourth step, based on the historical fusion features generated by the optimal solution and the real market indicator labels, trains a multi-task regression neural network as a dynamic market size assessment model. Utilizing its powerful nonlinear fitting and multi-objective collaborative learning capabilities, it achieves a reliable mapping from the high-dimensional fusion feature space to the core quantitative indicator space, completing the intelligent calculation from "data fusion" to "indicator evaluation". Finally, to achieve the automation of the evaluation process and the interpretability of the results, the fifth step loads and configures the optimal solution and the trained evaluation model to form a reusable online evaluation service. It automatically performs standardized fusion and model inference on data in new time windows and outputs a complete report containing dynamic evaluation values ​​and the logic of the fusion solution used, forming a closed loop of "solution solidification - feature fusion - model prediction - result presentation". This ensures that each evaluation is based on the same optimal and stable logic, thereby greatly improving the reproducibility, comparability, and decision support value of the evaluation results while obtaining real-time insights.

[0053] The acquisition of multi-source heterogeneous data of the target evaluation market, and the division of the multi-source heterogeneous data into multiple data modality sets based on data modality, specifically includes:

[0054] Continuous data with the same data structure and semantic expression in multi-source heterogeneous data are classified into the same data modality set. All data modality sets in multi-source heterogeneous data are obtained to complete the data modality division. The data structure and semantic expression include time-series numerical sequences, unstructured text and relational graph data.

[0055] The modal classification of multi-source heterogeneous data is essentially a homogenization based on the inherent structural features and semantic expression logic of the data. The aim is to design differentiated preprocessing and feature engineering strategies for different types of data, avoiding a "one-size-fits-all" approach that could lead to information loss or noise amplification. The reason for classifying data structures and semantic expressions into three categories—time-series numerical sequences, unstructured text, and relational graph data—is that these three types cover the vast majority of data source types in market size assessment scenarios, and their structural and semantic features differ significantly. The core feature of time-series numerical sequences is the structured expression of timestamps and continuous / discrete numerical values. Their semantics directly reflect the changing patterns of indicators over time (e.g., monthly sales, number of charging piles built, price indices). This type of data has a regular structure and clear semantics, allowing for direct extraction of features such as trends, cycles, and growth rates through statistical analysis, making it the core data support for quantitative market size assessment. The core feature of unstructured text is the unstructured expression of natural language character streams, with its semantics implied within the text content (e.g., user reviews on social media, patents). (Regarding the technological breakthroughs in the literature), this type of data lacks a fixed format and cannot be directly calculated numerically. It requires natural language processing techniques (such as sentiment analysis, topic extraction, and entity recognition) to transform semantics into quantifiable features, serving as crucial supplementary data for uncovering market size drivers. Relationship graph data is characterized by a network structure of entity-relationship-entity relationships. Its semantics reflect the logical connections between market participants (e.g., the supply chain relationship between automakers and battery suppliers, the marketing partnership between brands and livestreamers, and the supply and demand relationship between products and consumers). This type of data forms a topological network, and its semantics reflect the underlying relational structure of the market. Graph theory algorithms (such as centrality analysis and path analysis) can be used to extract the influence features of key nodes, making it a unique data source for assessing market structure stability and growth potential. This classification of three types of data ensures both the completeness of data classification (covering core data sources for market assessment) and the mutual exclusivity of classifications (the same data will not belong to two different modalities simultaneously), laying the foundation for subsequent targeted feature extraction and fusion, and ensuring that the value of data from different modalities can be fully explored.

[0056] Reference Figure 2 As shown, the process of generating at least one data fusion and feature construction scheme based on core market indicators and all data modal sets specifically includes:

[0057] The core market indicators for market size are set, and the core market indicators include at least: market size, annual incremental size, market share and market growth rate.

[0058] Based on the data structure and semantic expression, a pre-defined feature engineering strategy library is constructed.

[0059] Based on each core market metric, at least one feature extraction strategy is determined for each data modality set from a pre-defined feature engineering strategy library;

[0060] The combination of all feature extraction strategies corresponding to each data modality set is used as the set of candidate feature strategies for that modality.

[0061] Based on the historical information entropy of each data modality set and the Pearson correlation coefficient of the core market indicators, the initial modality fusion weights are obtained through a weighted synthesis formula.

[0062] Select a feature extraction strategy from the set of candidate feature strategies corresponding to each set of data modalities, and assign an initial modality fusion weight to combine them into a data fusion and feature construction scheme;

[0063] By iterating through all data modal sets and combining the feature extraction strategies with the initial fusion weights, several data fusion and feature construction schemes are obtained.

[0064] It can be explained that the data fusion and feature construction scheme is the bridge connecting the original data and the final evaluation model. Its quality directly determines the evaluation effect. The core of this scheme is to systematically construct the scheme space and quantify it into specific indicators and mapping rules that can be recognized by computers. This transforms the task of designing features and fusion weights into a search and optimization problem in a well-defined and finite combination space, making automated optimization possible. Its specific implementation process and specific implementation cases include:

[0065] The core quantitative indicators for assessing market size should include at least: market size (K1), annual incremental size (K2), market share (K3), and market growth rate (K4), with the specific calculation formulas as follows:

[0066] K1: Market size = Cumulative sales volume × Average product price;

[0067] K2: Annual incremental scale = Total cumulative sales in the current year - Total cumulative sales in the previous year;

[0068] K3: Market share = (Company's own market sales / Total industry sales) × 100%;

[0069] K4: Market growth rate = (Current period sales - Same period last year sales) / Same period last year sales × 100%;

[0070] The preset feature engineering strategy library is stored in the form of a queryable data table. It defines standardized feature extraction functional modules for different data modalities. Each functional module is encapsulated through predefined input data formats, processing logic, and output feature reduction. Its specific implementation can be based on well-known algorithms or mature tools in this field. The core innovation of this solution lies in the framework for dynamically selecting, weighting, and optimizing the combination of these functional modules, rather than the specific algorithm implementation within the modules. An example of the strategy library is shown in Table 1 below:

[0071] Table 1 Examples of Preset Feature Engineering Strategy Library

[0072] The strategy library defined in Table 1 embodies the core design philosophy of standardizing and modularizing the feature engineering process in this scheme. The evaluation framework constructed in this scheme provides a systematic method for dynamic selection, weight adaptation, and scheme optimization through automated processes based on such a predefined functional module library. Each functional module (S11-S33) in the predefined strategy library defines a clear and computable feature transformation logic. For example, the algorithm principles and implementation methods based on strategies S21 (TF-IDF), S22 (semantic embedding based on Transformer architecture), and S31 (node ​​degree centrality) belong to mature technologies in the fields of data processing and machine learning, and are supported by a large number of public documents and software libraries. Based on the objectives and input data of each functional module disclosed in this scheme, According to the requirements and output feature specifications (as shown in Table 1), these modules can be implemented by directly selecting or combining corresponding methods from existing technologies. For clarity, several non-restrictive examples are provided below: "Emotional polarity score (S23)" can be implemented using dictionary matching or a sentiment analysis model based on supervised learning training; "Policy support coefficient (S24)" can be implemented using weighted statistics of preset keywords or quantitative scoring through a text classification model. The choice of these specific implementation methods does not affect the operation of the core framework of this solution. The establishment of the strategy library enables the computer to transform the complex multi-source heterogeneous data processing task into a problem of selecting and combining a series of functional modules with clear interface specifications, thereby laying a solid foundation for subsequent automated solution generation and global optimization.

[0073] Specific examples of determining at least one feature extraction strategy for each data modality set from a preset feature engineering strategy library may include:

[0074] For K1 (existing scale): M1 candidate strategy = {S11}, M2 candidate strategy = {S24}, M3 candidate strategy = {S31};

[0075] For K4 (growth rate): M1 candidate strategy = {S12, S13}, M2 candidate strategy = {S23}, M3 candidate strategy = {S32};

[0076] Among them, the matching rules for K2 and K3 are similar, prioritizing strategies that are strongly correlated with the indicators;

[0077] The modality fusion weights are determined by pre-allocating them based on the historical information entropy of each data modality set and the Pearson correlation coefficient of the core market indicators using a weighted synthesis formula.

[0078] The weighted synthesis formula is as follows:

[0079] In the formula, For the first Pre-assigned fusion weights for each data modality For the first Information entropy calculated from historical data of each data modality This indicates that it is normalized (e.g., Min-Max is normalized to [0,1]). This is a global deterministic score; a higher score indicates more reliable modality data and should be assigned a higher weight. For the first The absolute values ​​of the Pearson correlation coefficients between candidate features extracted from each data modality and core market indicators. This is the relevance score; the higher the score, the more relevant the modality feature is to the target. For an adjustable equilibrium hyperparameter ( This is used to control the relative importance of deterministic and relevance scores in weighting. In the initial settings, it can be set... To ensure equal treatment, all calculated initial modal fusion weights are finally normalized so that their sum equals 1, yielding the formal initial fusion weights for this candidate scheme. ;

[0080] It should be further explained that the determination of the modal fusion weights is based on generating a candidate solution using the market stock size (K1) as an example:

[0081] Modus M1 (sales time series) selected strategy S11 (N-month moving average);

[0082] Modus M2 (policy text) selected strategy S24 (policy support coefficient);

[0083] Modality M3 (relationship graph) selected strategy S31 (node ​​degree centrality);

[0084] Based on the original sales sequence of M1, the original text set of M2, and the original graph data of M3, calculate their respective information entropy;

[0085] The corresponding historical data are processed using S11, S24, and S31 respectively to obtain the N-month moving average sales series, the monthly policy support coefficient series, and the monthly node degree centrality series.

[0086] Calculate the absolute values ​​of the Pearson correlation coefficients between these three sequences and the true value sequence of market stock size (K1) over the past L months;

[0087] The true value sequence of the market stock size (K1) is usually derived from publicly available data sources such as authoritative industry reports;

[0088] The obtained information entropies are normalized to obtain The absolute values ​​of the obtained correlation coefficients are normalized to obtain... ;

[0089] Based on the weighted composition formula, set The pre-allocated fusion weights for modes M1, M2, and M3 are calculated respectively.

[0090] The pre-assigned fusion weights of modes M1, M2, and M3 are normalized so that their sum is 1, thus obtaining the initial fusion weights of the scheme.

[0091] The data fusion and feature construction scheme is presented in the following form: K1: M1(S11)+ M2 (S24) + M3 (S31)

[0092] Wherein, M1(S11) represents the features of the time-series numerical sequence mode (M1) after processing with strategy S11 (moving average), M2(S24) represents the features of the policy text mode (M2) after processing with strategy S24 (policy support coefficient), and M3(S31) represents the features of the relationship graph mode (M3) after processing with strategy S31 (node ​​degree centrality). , , The initial fusion weights corresponding to modes M1, M2, and M3 are... .

[0093] Reference Figure 3 As shown, the step of calculating the feature stability index under each data fusion and feature construction scheme according to the generated data fusion and feature construction scheme specifically includes:

[0094] For each generated data fusion and feature construction scheme, the historical data within the preset historical evaluation window is divided into several fixed time windows. Modal features of each fixed time window are extracted through the preset feature extraction strategy in the scheme.

[0095] Based on the initial fusion weights, local determinism scores, and modal characteristics of each scheme within a fixed time window, the contribution of each modal information within that fixed time window is calculated.

[0096] For each fixed time window, the information contribution of all modalities within that window is integrated into the feature vector of the fusion scheme for that window;

[0097] Calculate the coefficient of variation of the feature vectors corresponding to all fixed time windows to form a coefficient of variation sequence;

[0098] The coefficient of variation refers to the ratio of the standard deviation of the eigenvector to the mean. The smaller the ratio, the more balanced the contribution of each mode is distributed and the smaller the fluctuation in the time slice.

[0099] The average value of the coefficient of variation sequence is the characteristic stability index of the scheme.

[0100] It can be explained that, in the process of generating data fusion and feature construction schemes, the initial fusion weights are obtained by weighted synthesis based on the global information entropy of each data modality (the distribution entropy of the original data in the entire historical evaluation window) and the Pearson correlation coefficient of the core market indicators, and are a phased global fixed quantity; the deterministic scores are divided into two categories, of which the global deterministic scores are obtained by normalizing the global information entropy and are used for calculating the initial fusion weights, and the local deterministic scores are obtained by normalizing the local information entropy of the original data in each historical time window and are used to correct for fluctuations in the quality of the current data. However, to ensure that the constructed feature representations remain relatively stable and consistent across different historical time windows (such as different months), and to avoid instability in the output of the subsequently trained evaluation model due to fluctuations in the features themselves, this scheme introduces a feature stability index as a quantitative tool. The calculation of this index begins with the information contribution of each data modality. This contribution is jointly determined by the initial fusion weights, the local determinism score of the corresponding time window, and the current modality feature value. Within each historical time window, the modality contributions calculated according to the selected scheme are concatenated into a fusion scheme feature vector for that window, representing the market state of that time window. Subsequently, the coefficient of variation (the ratio of standard deviation to mean) of each feature vector is calculated. This coefficient reflects the relative dispersion of the modality contribution distribution at that moment. The scheme results in inconsistent contributions of different modalities over time (i.e., some modalities contribute extremely high at specific times and extremely low at other times), leading to large fluctuations in the coefficient of variation sequence. This indicates that the feature representation constructed by the scheme is inherently unstable. By averaging the coefficients of variation across all time windows within the entire historical evaluation window, a feature stability index is obtained. The smaller the index value, the smoother and more consistent the feature representation generated by the scheme is in historical backtesting, and the less affected it is by short-term data noise or random fluctuations. Therefore, selecting a scheme with a smaller feature stability index can provide a more reliable and robust input foundation for subsequent training and evaluation models, thereby ultimately improving the reliability and repeatability of the dynamic market size evaluation results. The expression for calculating the contribution of each modal information within this fixed time window is as follows:

[0101] In the formula, For the first The first time window Contribution of data modality information For locally deterministic fractions, For the first The first time window Data modal features.

[0102] The process of constructing a dynamic market size assessment model based on a multi-task regression neural network architecture, according to the stability index of the scheme characteristics, and outputting predicted values ​​of core quantitative indicators specifically includes:

[0103] Based on the stability index of the scheme features, the scheme with the smallest index value is selected as the optimal data fusion and feature construction scheme, and the modality fusion weight and feature extraction strategy corresponding to the scheme are extracted.

[0104] A neural network architecture with multi-task regression is adopted, with the feature vector of the fusion scheme generated by the optimal solution as the model input and the predicted values ​​of four core quantitative indicators as the model output.

[0105] The neural network architecture for the multi-task regression consists of a shared low-level feature encoding network and four independent regression output heads;

[0106] Collect and record historical market size data, and obtain the historical fusion scheme feature vectors according to the feature vector extraction method of the fusion scheme in the optimal data fusion and feature construction scheme.

[0107] Obtain actual market size survey data for the same time window, and calculate the true value of the core quantitative indicator corresponding to that time window based on the quantitative formula of the indicator.

[0108] Based on the obtained historical fusion scheme feature vectors and the true values ​​of core quantitative indicators, a training sample set is constructed and divided into a training set, a validation set, and a test set in an 8:1:1 ratio.

[0109] Preprocessing of the training sample set includes at least: data normalization, outlier removal, and interpolation;

[0110] Based on the mean square error formula, the sum of squared deviations between the predicted and actual values ​​of each core quantitative indicator is quantified, and a loss function is constructed.

[0111] With the goal of minimizing the loss function, the Adam optimizer is used to train the model using the training set data, and an early stopping mechanism is introduced to prevent overfitting.

[0112] After training, its parameters are fixed to obtain a deployable dynamic market size assessment model, and the model is used to output predicted values ​​of core quantitative indicators.

[0113] It can be explained that the optimal data fusion and feature construction scheme solves the problem of "how to stably extract and fuse information from multi-source data". However, the feature vector of the fusion scheme output is not the market size value directly used. Therefore, this scheme introduces a multi-task regression neural network and uses its powerful nonlinear fitting and multi-objective collaborative learning capabilities to construct a mapping model from the fusion feature space to the specific quantitative indicator space, that is, the dynamic evaluation model of market size. In the deployment phase, it is only necessary to process the original data of the latest time window according to the optimal scheme, generate the fusion scheme feature vector and input it into the trained model, so as to obtain the dynamic evaluation results of the current market size core indicators in real time, thereby realizing the deep quantification and timely insight into the market status.

[0114] The preprocessing specifically includes: truncating the abnormal contributions of each dimension in the vector based on the 3σ principle; filling missing time windows using time series linear interpolation; and normalizing the true values ​​of the four continuous indicators using Z-score to eliminate dimensional differences and accelerate model training convergence.

[0115] The loss function expression is as follows: Loss = 1 N ∑ t = 1 N ∑ i = 1 4 [ K ^ i ( t ) − K i ( t )] 2

[0116] In the formula, To account for the overall prediction error, The number of training samples, For the first The first time window Predicted values ​​of key quantitative indicators, For the first The first time window The true values ​​of the core quantitative indicators.

[0117] The process of obtaining a dynamic market size assessment report based on the optimal data fusion and feature construction scheme and the dynamic market size assessment model specifically includes:

[0118] Load and configure the optimal data fusion and feature construction scheme and the trained market size dynamic evaluation model;

[0119] Specifically, the optimal solution is associated and bound with a pre-defined feature engineering strategy library to determine the feature extraction logic for each data modality; the evaluation model is loaded with its fixed parameters and placed in an executable state.

[0120] Each time a dynamic market size assessment is performed, the following standardized process is executed based on the loaded optimal data fusion and feature construction scheme:

[0121] For the raw data of the current time window of each data modality, the feature extraction strategy specified in the scheme is invoked to extract the feature values ​​of each modality;

[0122] Based on the initial fusion weights and local deterministic scores, combined with the extracted modal feature values, the information contribution of each modality in the current time window is calculated;

[0123] The information contribution values ​​of all modalities are concatenated and preprocessed to generate a standardized fusion scheme feature vector;

[0124] Input the generated standardized fusion scheme feature vector into the loaded market size dynamic evaluation model;

[0125] The model uses forward inference to simultaneously output dynamic evaluation values ​​for four core quantitative indicators in the current time window: market stock size, annual incremental size, market share, and market growth rate.

[0126] The dynamic evaluation values ​​of the four core quantitative indicators, along with the presentation of the data fusion and feature construction scheme, are output together to generate an evaluation report.

[0127] This solution's technical architecture is clearly divided into two phases: "offline optimization training" and "online deployment and operation." In the offline phase, the optimal data fusion and feature construction scheme (the decision-making brain) selected through stability assessment, along with the market size dynamic assessment model (the computing engine) trained using historical data, together form the core foundation of the assessment task. However, these offline results are essentially static parameter rules and model weights, which cannot directly process real-time new data. Therefore, it is necessary to instantiate the static offline results into a callable online service through the key operation of "loading and configuring the optimal scheme and the assessment model together." In this process, the optimal scheme is bound to a preset feature engineering strategy library, solidifying the feature extraction logic of each modality. The assessment model loads the solidified parameters and enters the inference state, effectively reducing the arbitrariness of the assessment process and ensuring that each dynamic assessment follows the same set of historically validated optimal logic, guaranteeing the consistency, comparability, and reproducibility of the output results across different time windows.

[0128] Furthermore, the method according to the embodiments of this application can also be achieved by means of... Figure 4 The architecture of the electronic device shown is used to implement this. For example... Figure 4As shown, the electronic device 500 may include a bus 501, one or more CPUs 502, a read-only memory (ROM) 503, a random access memory (RAM) 504, a communication port 505 connected to a network, an input / output component 506, a hard disk 507, etc. The storage device in the electronic device 500, such as the ROM 503 or the hard disk 507, may store the market size dynamic evaluation model construction method based on multi-source heterogeneous data fusion provided in this application. The electronic device 500 may also include a user interface 508. Of course, Figure 4 The architecture shown is merely exemplary and can be omitted as needed when implementing different devices. Figure 4 One or more components in the illustrated electronic device.

[0129] Figure 5 This is a schematic diagram of a computer-readable storage medium structure provided in one embodiment of this application. Figure 5 The diagram illustrates a computer-readable storage medium 600 according to one embodiment of this application. The computer-readable storage medium 600 stores computer-readable instructions. When executed by a processor, the computer-readable instructions can perform a method for constructing a dynamic market size assessment model based on multi-source heterogeneous data fusion, as described above with reference to the accompanying drawings, according to an embodiment of this application. The storage medium 600 includes, but is not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0130] In summary, the advantages of this invention are: by introducing a feature stability index to automatically optimize and solidify the multi-source data fusion scheme, the temporal consistency of the input features of the evaluation model is ensured, thereby fundamentally improving the robustness, repeatability, and interpretability of the dynamic market size evaluation results.

[0131] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.

Claims

1. A method for constructing a dynamic market size assessment model based on multi-source heterogeneous data fusion, characterized in that, include: Acquire multi-source heterogeneous data of the target evaluation market, and divide the multi-source heterogeneous data into multiple data modality sets based on data modality; Based on core market metrics and all data modal sets, generate at least one data fusion and feature construction scheme; Based on the generated data fusion and feature construction schemes, calculate the scheme feature stability index for each data fusion and feature construction scheme. Based on the stability index of the scheme characteristics, a dynamic market size assessment model is constructed based on a multi-task regression neural network architecture, and the predicted values ​​of core quantitative indicators are output. Based on the optimal data fusion and feature construction scheme and the dynamic market size assessment model, a dynamic market size assessment report is obtained.

2. The method for constructing a dynamic market size assessment model based on multi-source heterogeneous data fusion according to claim 1, characterized in that, The acquisition of multi-source heterogeneous data of the target evaluation market, and the division of the multi-source heterogeneous data into multiple data modality sets based on data modality, specifically includes: Continuous data with the same data structure and semantic expression in multi-source heterogeneous data are classified into the same data modality set. All data modality sets in multi-source heterogeneous data are obtained to complete the data modality division. The data structure and semantic expression include time-series numerical sequences, unstructured text and relational graph data.

3. The method for constructing a dynamic market size assessment model based on multi-source heterogeneous data fusion according to claim 2, characterized in that, The specific steps for generating at least one data fusion and feature construction scheme based on core market indicators and all data modal sets include: The core market indicators for market size are set, and the core market indicators include at least: market size, annual incremental size, market share and market growth rate. Based on the data structure and semantic expression, a pre-defined feature engineering strategy library is constructed. Based on each core market metric, at least one feature extraction strategy is determined for each data modality set from a pre-defined feature engineering strategy library; The combination of all feature extraction strategies corresponding to each data modality set is used as the set of candidate feature strategies for that modality. Based on the historical information entropy of each data modality set and the Pearson correlation coefficient of the core market indicators, the initial modality fusion weights are obtained through a weighted synthesis formula. Select a feature extraction strategy from the set of candidate feature strategies corresponding to each set of data modalities, and assign an initial modality fusion weight to combine them into a data fusion and feature construction scheme; By iterating through all data modal sets and combining the feature extraction strategies with the initial fusion weights, several data fusion and feature construction schemes are obtained.

4. The method for constructing a dynamic market size assessment model based on multi-source heterogeneous data fusion according to claim 3, characterized in that, The step of calculating the feature stability index under each generated data fusion and feature construction scheme specifically includes: For each generated data fusion and feature construction scheme, the historical data within the preset historical evaluation window is divided into several fixed time windows. Modal features of each fixed time window are extracted through the preset feature extraction strategy in the scheme. Based on the initial fusion weights, local determinism scores, and modal characteristics of each scheme within a fixed time window, the contribution of each modal information within that fixed time window is calculated. For each fixed time window, the information contribution of all modalities within that window is integrated into the feature vector of the fusion scheme for that window; Calculate the coefficient of variation of the feature vectors corresponding to all fixed time windows to form a coefficient of variation sequence; The coefficient of variation refers to the ratio of the standard deviation of the eigenvector to the mean. The smaller the ratio, the more balanced the contribution of each mode is distributed and the smaller the fluctuation in the time slice. The average value of the coefficient of variation sequence is the characteristic stability index of the scheme.

5. The method for constructing a dynamic market size assessment model based on multi-source heterogeneous data fusion according to claim 4, characterized in that, The process of constructing a dynamic market size assessment model based on a multi-task regression neural network architecture, according to the stability index of the scheme characteristics, and outputting predicted values ​​of core quantitative indicators specifically includes: Based on the stability index of the scheme features, the scheme with the smallest index value is selected as the optimal data fusion and feature construction scheme, and the modality fusion weight and feature extraction strategy corresponding to the scheme are extracted. A neural network architecture with multi-task regression is adopted, with the feature vector of the fusion scheme generated by the optimal solution as the model input and the predicted values ​​of four core quantitative indicators as the model output. The neural network architecture for the multi-task regression consists of a shared low-level feature encoding network and four independent regression output heads; Collect and record historical market size data, and obtain the historical fusion scheme feature vectors according to the feature vector extraction method of the fusion scheme in the optimal data fusion and feature construction scheme. Obtain actual market size survey data for the same time window, and calculate the true value of the core quantitative indicator corresponding to that time window based on the quantitative formula of the indicator. Based on the obtained historical fusion scheme feature vectors and the true values ​​of core quantitative indicators, a training sample set is constructed and divided into a training set, a validation set, and a test set in an 8:1:1 ratio. Preprocessing of the training sample set includes at least: data normalization, outlier removal, and interpolation; Based on the mean square error formula, the sum of squared deviations between the predicted and actual values ​​of each core quantitative indicator is quantified, and a loss function is constructed. With the goal of minimizing the loss function, the Adam optimizer is used to train the model using the training set data, and an early stopping mechanism is introduced to prevent overfitting. After training, its parameters are fixed to obtain a deployable dynamic market size assessment model, and the model is used to output predicted values ​​of core quantitative indicators.

6. The method for constructing a dynamic market size assessment model based on multi-source heterogeneous data fusion according to claim 5, characterized in that, The process of obtaining a dynamic market size assessment report based on the optimal data fusion and feature construction scheme and the dynamic market size assessment model specifically includes: Load and configure the optimal data fusion and feature construction scheme and the trained market size dynamic evaluation model; Specifically, the optimal solution is associated and bound with a pre-defined feature engineering strategy library to determine the feature extraction logic for each data modality; the evaluation model is loaded with its fixed parameters and placed in an executable state. Each time a dynamic market size assessment is performed, the following standardized process is executed based on the loaded optimal data fusion and feature construction scheme: For the raw data of the current time window of each data modality, the feature extraction strategy specified in the scheme is invoked to extract the feature values ​​of each modality; Based on the initial fusion weights and local deterministic scores, combined with the extracted modal feature values, the information contribution of each modality in the current time window is calculated; The information contribution values ​​of all modalities are concatenated and preprocessed to generate a standardized fusion scheme feature vector; Input the generated standardized fusion scheme feature vector into the loaded market size dynamic evaluation model; The model uses forward inference to simultaneously output dynamic evaluation values ​​for four core quantitative indicators in the current time window: market stock size, annual incremental size, market share, and market growth rate. The dynamic evaluation values ​​of the four core quantitative indicators, along with the presentation of the data fusion and feature construction scheme, are output together to generate an evaluation report.

7. An electronic device, characterized in that, include: At least one processor; And, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform a method for constructing a dynamic market size assessment model based on multi-source heterogeneous data fusion as described in any one of claims 1-6.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the method for constructing a dynamic market size assessment model based on multi-source heterogeneous data fusion as described in any one of claims 1-6.

Images