Data construction method and device based on industrial chain atlas and related medium

Abstract

The invention discloses a data construction method and device based on an industrial chain atlas and a related medium. The method comprises the following steps: respectively importing structured data and unstructured data of a plurality of data sources into a data warehouse; performing standardized marking on the structured data based on a predetermined marking rule, and performing standardized marking on the unstructured data by using a natural language processing model to obtain an industrial tag; constructing an industrial chain atlas, and mounting the industrial label to the industrial chain atlas; and performing data association on the industrial labels based on the industrial chain atlas to obtain a multi-level relational data structure. According to the method, standardized marking is carried out on the structured data and the unstructured data, then the obtained industrial labels are mounted to the industrial chain atlas, and then data association is carried out on the industrial labels, so that the efficiency of an industrial research tool in the aspect of knowledge sharing collaboration is improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data construction method, device and related media based on an industrial chain graph. Background Art

[0002] Existing industry research tools have significant defects, which are mainly manifested in the following aspects: First, the industry chain is diverse and lacks unified standards, which leads to inconsistent perspectives of researchers when defining upstream and downstream relationships, affecting the accuracy of data integration and analysis. Second, data is difficult to obtain and easy to miss, especially indicator data, research reports, news, etc. are scattered in different sources and cannot be managed centrally. Third, the scope of comparable companies is not clearly defined, the industry classification is too broad, and there is a lack of a refined classification system, which leads to distorted data and affects the reliability of research conclusions.

[0003] In addition, existing tools still rely on a lot of manual operations to update data, which is cumbersome and error-prone, making it difficult to achieve real-time updates, reducing research efficiency. Research models and results are scattered and lack a unified collaboration platform, making it difficult for team members to effectively share knowledge and collaborate, affecting overall work efficiency. The lack of breadth and depth of information tracking, especially the weak ability to monitor long-tail information and potential market dynamics in real time, further limits the application of research tools in dynamic markets.

[0004] In summary, current industry research tools have obvious shortcomings in knowledge sharing and collaboration. Summary of the invention

[0005] The embodiments of the present invention provide a data construction method, device and related media based on the industrial chain map, aiming to solve the problem that current industrial research tools are inefficient in knowledge sharing and collaboration.

[0006] In a first aspect, an embodiment of the present invention provides a data construction method based on an industrial chain graph, comprising:

[0007] Import structured data and unstructured data from multiple data sources into the data warehouse respectively;

[0008] The structured data is standardized and labeled based on a predetermined labeling rule, and the unstructured data is standardized and labeled using a natural language processing model to obtain an industry label;

[0009] Construct an industrial chain map, and attach the industrial label to the industrial chain map;

[0010] The industry tags are data-associated based on the industry chain graph to obtain a multi-level relational data structure.

[0011] In a second aspect, an embodiment of the present invention provides a data construction device based on an industrial chain graph, including:

[0012] A data import unit, used to import structured data and unstructured data from multiple data sources into the data warehouse respectively;

[0013] A data labeling unit, used to perform standardized labeling on the structured data based on a predetermined labeling rule, and to perform standardized labeling on the unstructured data using a natural language processing model to obtain an industry label;

[0014] A data mounting unit, used to construct an industrial chain map and mount the industrial label to the industrial chain map;

[0015] The data association unit is used to perform data association on the industry tags based on the industry chain map to obtain a multi-level relational data structure.

[0016] In a third aspect, an embodiment of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the data construction method based on the industrial chain map of the first aspect when executing the computer program.

[0017] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the data construction method based on the industrial chain map of the first aspect is implemented.

[0018] The embodiment of the present invention provides a data construction method based on the industrial chain map, including importing structured data and unstructured data from multiple data sources into a data warehouse respectively; standardizing and marking the structured data based on a predetermined marking rule, and standardizing and marking the unstructured data using a natural language processing model to obtain industry labels; constructing an industrial chain map, and mounting the industry labels to the industrial chain map; and data-associating the industry labels based on the industrial chain map to obtain a multi-level relational data structure. The present invention improves the efficiency of industrial research tools in knowledge sharing and collaboration by standardizing and marking structured data and unstructured data, mounting the obtained industry labels to the industrial chain map, and then data-associating the industry labels.

[0019] The embodiments of the present invention also provide a data construction device, a computer device and a storage medium based on the industrial chain map, which also have the above-mentioned beneficial effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0021] Figure 1 A schematic diagram of a process of constructing data based on an industrial chain graph provided by an embodiment of the present invention;

[0022] Figure 2 An application flow chart of a data construction method based on an industrial chain graph provided by an embodiment of the present invention;

[0023] Figure 3 A schematic diagram of an application of a data construction method based on an industrial chain graph provided by an embodiment of the present invention;

[0024] Figure 4 A schematic block diagram of a data construction device based on an industrial chain map provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0027] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0028] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0029] See below Figure 1 , Figure 1 A flow chart of a data construction method based on an industrial chain map provided in an embodiment of the present invention specifically includes steps S101 to S104.

[0030] S101, importing structured data and unstructured data from multiple data sources into a data warehouse respectively;

[0031] S102, performing standardized labeling on the structured data based on a predetermined labeling rule, and performing standardized labeling on the unstructured data using a natural language processing model to obtain an industry label;

[0032] S103, constructing an industrial chain map, and attaching the industrial label to the industrial chain map;

[0033] S104: Perform data association on the industry tags based on the industry chain graph to obtain a multi-level relational data structure.

[0034] In step S101, various types of data from multiple data sources are first collected and organized. These data may include structured data (such as corporate financial reports, market data, product sales data, etc.) and unstructured data (such as news, research reports, policy texts, etc.). After all data are uniformly formatted and cleaned, they will be imported into the data warehouse for subsequent processing and analysis. Structured data is usually stored in a table format, while unstructured data includes natural language text data.

[0035] In one embodiment, the step S101 includes:

[0036] Extract the structured data and unstructured data from multiple data sources through ETL tools, and perform data cleaning;

[0037] The cleaned data are imported into different tables of the data warehouse using database connection technology to obtain basic data.

[0038] In this embodiment, ETL (Extract-Transform-Load) tools play a key role in the data warehouse construction process. ETL tools are used to extract data from multiple data sources, which may include structured data sources (such as relational databases, Excel files, CSV files, etc.) and unstructured data sources (such as online news, research reports, policy documents, etc.). For example, suppose that data source A is the financial report of a listed company, and the structured data includes its financial indicators such as revenue, profit, cost, etc.; data source B is news related to the company published by a news website, including unstructured text data such as company trends and market analysis. The ETL tool will extract the required structured and unstructured data from these data sources and prepare for subsequent processing. After extracting the original data from each data source, data cleaning is required. The purpose of data cleaning is to remove dirty data (such as duplicate data, missing values, inconsistent data) and irrelevant data to ensure that the quality of the extracted data meets the requirements of subsequent analysis.

[0039] For structured data, data cleaning includes:

[0040] Deduplication: Remove duplicate records, such as the same financial report submitted multiple times during the same time period.

[0041] Filling missing values: For missing financial data (such as missing profit information for a certain quarter), you can use appropriate filling methods (such as using the mean or median) or delete incomplete records.

[0042] Data formatting: Ensure that all data is in the same unit and format. For example, unify all amounts into "10,000 yuan" or "1,000 yuan" to ensure that comparisons between data are valid.

[0043] For unstructured data, data cleaning usually includes:

[0044] Remove irrelevant information: For example, remove advertisements, date stamps, or content that is not related to the company's main business from news reports.

[0045] Word segmentation and denoising: For Chinese news texts, word segmentation tools (such as Jieba word segmentation) are used to break the text into words and remove useless words (such as stop words).

[0046] Text standardization: Standardize different expressions of the same meaning, for example, treat "Company AA" and "AA Company" as the same entity. That is, after data cleaning, "Company AA" and "AA Company" can be standardized as "Company AA" for subsequent analysis.

[0047] Furthermore, after the data cleaning is completed, the next step is to import the cleaned data into the data warehouse. At this time, the data is transferred to the data warehouse using database connection technology. The data warehouse is a centralized platform for storing the cleaned data, which supports efficient data query and analysis. In the data warehouse, different types of data will be stored in different tables for subsequent association and query.

[0048] Assume that multiple data tables have been defined in the data warehouse, for example:

[0049] Financial data table: stores the company's financial indicator data, including revenue, profit, cost, etc.

[0050] News data table: stores company-related news reports and analysis, mainly text data.

[0051] Industrial chain data table: stores the upstream and downstream relationship information of the company in the industrial chain.

[0052] In this process, the ETL tool uses database connection technology (such as JDBC or ODBC) to import the cleaned data into the corresponding database table. For example, the ETL tool imports the financial data extracted from data source A into the "financial data table" through JDBC connection, and imports the news text data extracted from data source B into the "news data table", and stores these data according to predefined fields.

[0053] In a data warehouse, the design of data tables needs to follow certain specifications to ensure that data can be stored and accessed efficiently. In an embodiment of the present invention, the design of data tables follows the following principles:

[0054] Field specification: Each data table should have a uniform field structure and data type. For example, in a "Financial Data Table", the fields may include "Company Name", "Reporting Time", "Revenue", "Profit", etc., and the field data type should be consistent with the data, such as the "Revenue" field is a numeric type.

[0055] Primary key design: In order to ensure the uniqueness of data in the data table, you need to design a suitable primary key. For example, the primary key of the "financial data table" can be a combination of "company name" and "report time" to ensure that each record is unique.

[0056] Index optimization: In order to speed up the query process, some fields in the data table (such as "Company Name", "Report Time", etc.) may need to be indexed to improve data retrieval efficiency.

[0057] Furthermore, after completing data import and cleaning, structured data and unstructured data have been stored in the data warehouse. Through the query interface or API in the data warehouse, users can access and extract the basic data (i.e. structured data and unstructured data) required for these industry research. The obtained structured data and unstructured data can be used for subsequent industrial chain map construction, data association and analysis processing.

[0058] In step S102, the imported structured data is standardized and labeled based on the predetermined labeling rules, and the unstructured data is analyzed using the natural language processing model to finally obtain industry labels. Industry labels include information such as product category, industry classification, company name, upstream and downstream relationships, etc., which can help establish the correlation between data.

[0059] In one embodiment, the step S102 includes:

[0060] Define the type labels of the data set; wherein the type labels include sentiment type, company type, text type and event type;

[0061] Using the type label to train the natural language processing model, and optimizing the accuracy of the natural language processing model by a hyperparameter tuning method to obtain a trained natural language processing model;

[0062] The trained natural language processing model is used to perform standardized labeling on the unstructured data to obtain the industry label.

[0063] In this embodiment, it is necessary to define type labels for text data extracted from different data sources (such as news, research reports, social media, etc.). Sentiment labels are used to identify the sentiment expressed in the text (such as positive, negative, neutral, etc.). Company type labels are used to identify the company names mentioned in the text and their corresponding categories. Text type labels are used to identify the type of text (such as news, research reports, announcements, etc.). Event type labels are used to identify the type of event described in the text.

[0064] After defining the type label, select a suitable natural language processing (NLP) model for training. Usually, a pre-trained language model based on deep learning can be selected, such as BERT, RoBERTa, GPT, etc. During the training process, the NLP model learns the type label, extracts text features from it, and learns how to predict each type label based on these features. During the training process, the performance of the model (such as recall rate, precision rate, F1 value, etc.) is affected by multiple hyperparameters, such as learning rate, batch size, optimization algorithm, etc. Therefore, in an embodiment of the present invention, a hyperparameter tuning method is also used to optimize the model performance. These hyperparameters are adjusted by methods such as grid search, random search or Bayesian optimization to find the best parameter combination, thereby improving the accuracy and prediction accuracy of the model. For example, during the training process, if it is found that the accuracy of positive emotions is low when the model recognizes the emotion type, the model performance can be optimized by adjusting the learning rate or increasing the proportion of training data, and finally a higher accuracy rate is obtained.

[0065] Furthermore, after the model training is completed, the trained NLP model is used to label the unstructured data. For example, when labeling a news article about "Guangdong Electric Power A obtains patent authorization: 'Lithium battery energy storage backup device'", the model analyzes the text content and identifies the following industry tags:

[0066] Emotion type: Positive;

[0067] Company: ABC Co., Ltd.

[0068] Product: Li-ion Battery

[0069] News type: Company;

[0070] Event type: Patent Licensed.

[0071] Ultimately, the labeled text will be output as standardized industry labels and stored in the data warehouse for subsequent analysis and query. These standardized labels will help researchers quickly understand the content and background of the data and its relationship with each node in the industry chain map.

[0072] After obtaining standardized industry labels, the subsequent steps will use these labels to associate data and build a multi-level relational data structure in combination with the industry chain map. In this way, different types of data (such as news, research reports, and financial reports) can be effectively organized and analyzed based on the upstream and downstream relationships of the industry chain, thereby providing data support for investment decisions.

[0073] In step S103, an industrial chain map is constructed based on the marked industry tags. The industrial chain map is a hierarchical graphical structure, which contains detailed descriptions of different industrial links, including standardized products, enterprises, upstream and downstream relationships, etc. of each link. Each node in the map represents an industrial link, and each node contains the core information of the link (such as product category, industry classification, company name, etc.). The industry tags obtained in step S102 are mounted to the corresponding nodes in the map according to the upstream and downstream relationships and classification standards of the industry chain to form a complete industrial chain map. At this point, the industrial chain map has connected the industry tags with each link of the industry chain through data tags.

[0074] In one embodiment, the step S103 includes:

[0075] Classify the imported products based on the RoBERTa classification algorithm to classify products of different categories into corresponding categories according to predefined industry standards and obtain industry data;

[0076] Performing standardized mapping on the industry data to adjust the uniformity between different representations of the same category of products to obtain standardized product data;

[0077] Based on the standardized product data and industry data, the knowledge graph technology is used to edit and generate the upstream and downstream supply relationships between products to obtain the industrial chain map.

[0078] In this embodiment, data imported from different data sources (such as financial data, product catalogs, supply chain information, etc.) are prepared. These data include information such as product name, description, category, price, sales volume, etc. In this step, the RoBERTa model is used to divide the imported products into several categories according to their characteristics (such as product description, product type, etc.). Of course, the K-means algorithm can also be used. Specifically, the execution process of the K-means algorithm is as follows:

[0079] Select relevant features of the product as input for clustering, such as product keywords (such as "mobile phone", "television"), price range, usage, etc. These features can be extracted from product descriptions using text analysis techniques;

[0080] Determine the number of clusters K. The K value indicates how many categories the product data is to be divided into. The optimal K value can be selected through the algorithm's internal evaluation indicators (such as silhouette coefficient, elbow rule, etc.). Suppose K = 5 is finally selected, which means that the products are divided into 5 categories.

[0081] By continuously calculating cluster centers and assigning each product to the closest cluster until convergence. In each iteration, the algorithm updates the center of each cluster and reallocates each product according to the new center;

[0082] The imported products are classified into different categories, and each category of products is connected to the predefined industry standards. The industry data obtained is the result of classification by the K-means clustering algorithm, which represents various products and the industry categories to which they belong. For example, if there are products such as "smartphones", "home TVs", and "game consoles" in the input product data, the K-means algorithm will classify these products into different categories such as "consumer electronics" and "household items". Preferably, the clustered results may not be perfectly matched with the industry standards. If there are predefined industry standards, they can be used directly, such as classification algorithms based on RoBERTa or other machine learning classification algorithms.

[0083] Furthermore, in actual industry data, products of the same category may be described in different ways. For example, "smartphone" may be described by different companies as "smart mobile phone", "smart terminal", etc., resulting in inconsistency in product names and descriptions. Therefore, standardized mapping is required to ensure that products of the same category have a unified identification to facilitate subsequent data processing and analysis. Specifically:

[0084] For example, "power lithium-ion battery" may be described by different companies as "power lithium battery", "lithium-ion battery", etc., and can be unified as "power lithium-ion battery";

[0085] Unify the description, purpose, and function of products of the same category based on information provided by industry standards and data sources. For example, for products in the "battery" category, no matter how different companies describe the product, it will eventually be mapped to a unified standardized product description, including its main functions, features, and technical parameters;

[0086] Through manual annotation, rule engines or machine learning technology, a mapping table is established to list various products and their standard names and descriptions. Through the standardized mapping process, the consistency of product categories and descriptions is adjusted to the industry standardization requirements, and finally standardized product data is obtained.

[0087] Furthermore, the knowledge graph technology can be JanusGraph, which is an open source graph database that can efficiently store and query graph structured data and is very suitable for processing complex related data. Each node in the industrial chain graph represents an industrial link, and each node is connected by a relationship, indicating the upstream and downstream supply relationship between products, companies, industries, etc. Based on standardized product data and industry data, the knowledge graph technology is used to generate the industrial chain graph. The specific steps are as follows:

[0088] Create a node. Each product, company, industry, and other information will be stored as a node in the graph database.

[0089] The nodes represent the connections between different industrial links through relationships.

[0090] When building the graph, the upstream and downstream relationships between product nodes are filled in based on industry data;

[0091] Through knowledge graph technology, standardized product data and industry data are finally combined to generate a complete industrial chain map. The map shows the supply and demand relationship between different products, companies and industries, helping researchers, investors and others to more intuitively understand the structure of the industrial chain, the interaction of various links and its impact on the market.

[0092] Combination Figure 2 As shown, in step S104, Figure 2 The IT personnel in the company first complete the push of basic data from the financial basic database, news and public opinion analysis data, etc., and also make relevant adjustments to the relevant data to construct an industrial chain map. By associating the data of each node in the industrial chain map, a multi-level relationship structure between the data is established, and the investment research team can apply it. Specifically, based on the products, companies and upstream and downstream relationships marked in the industrial chain map, the mutual relationship between different industrial links is further explored and analyzed. For example, different companies based on the same product label can analyze their position, relationship and mutual influence in the industrial chain to construct a multi-dimensional and multi-level relationship data structure. This data structure allows researchers to more intuitively analyze the interdependence between the various links in the industrial chain and their impact on the market.

[0093] In one embodiment, the step S104 includes:

[0094] Editing, formatting and optimizing the layout of the industry labels through the graphical interface of the industry chain map;

[0095] Automatically associating nodes in the industry tag according to the attribute editing information input by the user;

[0096] The upstream and downstream relationships between the nodes in the industry tags are customized through the graphic connection function.

[0097] In this embodiment, the user can edit the industry label in the industrial chain map on the provided graphical interface, including the content, format, layout, etc. of the node. The graphical interface presents the industrial chain map in a graphical way, so that the user can adjust and optimize the map by dragging, clicking, etc. In the graphical interface, the user can directly click on any node in the map to edit it. For example, if the user wants to modify the description of a product node, he can double-click or right-click on the node to enter the editing interface to modify the properties of the node, such as product name, standardized description, company information, etc. In the graphical interface, the user can also adjust the format of the industrial chain map to ensure that the map is clear, beautiful, and easy to understand. For example, the user can adjust the color, shape, font, etc. of each node to make the distinction between each node more obvious and enhance the visualization effect of the map. In addition, layout optimization is also very important, especially when there are a large number of nodes in the industrial chain map, a reasonable layout helps to improve the efficiency of information transmission and the readability of the map. The user can optimize the structure of the map by adjusting the arrangement of the nodes, selecting automatic layout or manually adjusting the distance and connection method between the nodes to make it more in line with actual use needs.

[0098] Furthermore, in the industrial chain map, nodes usually represent different links of the industrial chain, such as companies, products, industries, etc., and the relationship between nodes is represented by the connecting lines in the map. When editing the industry label, the user needs to associate the attributes between multiple nodes to form a complete information flow of the industrial chain. Users can enter specific attribute information (such as company name, product type, industry classification, etc.) through the attribute editing function of the graphical interface so that the system can automatically generate associations between nodes based on these attribute information. Based on the attribute information entered by the user, the system can automatically perform associations between nodes. Specifically, when the user enters a certain attribute (such as product name or company name), the system will automatically find matching nodes according to pre-defined rules and data models, and generate corresponding relationships. For example: after the user enters "BB" as the company name, the system will automatically find product nodes related to "BB" in the industrial chain map (such as "BB smartphone"). Then, the system automatically creates relationship nodes between "BB" and "smartphone" based on the input information, such as "production" or "supply" relationships. This process greatly improves the efficiency of constructing the industrial chain map. Users no longer need to manually select and associate nodes one by one. The system can intelligently complete the association between nodes based on the input information, reducing the possibility of manual operation and errors.

[0099] Furthermore, in the industrial chain map, each node establishes a logical relationship between the upstream and downstream supply chains through relationships (or connecting lines). In order to better demonstrate the interaction between products, companies, industries and other links in the industrial chain, users can use the graphic connection function to manually or automatically add upstream and downstream relationships between nodes. Through the graphic connection function, users can customize the upstream and downstream relationships between nodes in the industry label in the map. Users can manually define the relationship between them by simply clicking on a node and then dragging the connection line to another node. Relationship types can include:

[0100] Supply relationship: such as "Company A supplies electronic components to Company B";

[0101] Demand relationship: such as “consumers demand smartphones”;

[0102] Partnership: For example, “Company A and Company B collaborate to develop new products.”

[0103] For example, in the industry chain graph, users may connect the "smartphone" node with the "chip supplier" node to indicate that the "smartphone" is provided with chips by the "chip supplier". At this time, users can select the relationship type as "supply" or "cooperation" to intuitively express the upstream and downstream relationships through graphical operations.

[0104] Combination Figure 3 As shown in the figure, the graphical connection function not only supports users to manually define and adjust relationships, but also automatically optimizes the connection method according to the structure and layout of the graph to ensure that the relationship between nodes is clear and visible. For example, when the user adds multiple connection lines, the system will automatically adjust the layout of the connection lines to avoid too many line crossings, ensuring the beauty and readability of the graph. Figure 3 The industry chain map of lithium-ion batteries is shown in the figure. Through the graphic connection function, the industry chain map can not only present the basic information of each industry link, but also clearly show the dynamic interaction between each link, providing users with tools for in-depth analysis of the operation of the industry chain.

[0105] In one embodiment, after step S104, the following steps are included:

[0106] Based on the preset industry dashboard building function, select the directory structure and configure permissions according to the needs of industry analysis;

[0107] Selecting the structured data through a data browser interface based on the directory structure, and creating an analysis chart using the structured data;

[0108] Arrange the analysis charts in order to the industry dashboard to obtain a dashboard analysis framework;

[0109] The latest indicator changes of the structured data are obtained, and the dashboard analysis framework is updated in real time.

[0110] In this embodiment, the industry dashboard building function is a tool used to help users customize and display industry data analysis frameworks according to industry analysis needs. Through this function, users can select appropriate directory structures according to the needs of specific industries and configure corresponding permissions to ensure the security and effectiveness of data display and analysis. The directory structure refers to the way information is organized in the industry dashboard, which determines the order and level of display of data and analysis charts in the dashboard. Users can choose different directory structures according to actual needs. For example, users can build directory structures based on dimensions such as products, companies, and regions to help analysts better organize and access data at different levels. Permission configuration is a necessary step to ensure that different users access relevant data and analysis charts according to their roles. According to different industry analysis needs, users need to set different access rights. For example, some data may need to be accessible only to senior users, while some analysis charts can be viewed by all team members. If a "Financial Statement Analysis" chart is sensitive information and only senior management is allowed to view it, permissions can be configured to restrict ordinary employees from accessing the analysis chart. Permission configuration can include permission levels such as "read-only", "edit" and "administrator".

[0111] Furthermore, after the directory structure is determined, the user can select the corresponding structured data through the data browser interface. The data browser interface provides an interactive platform where users can select structured data related to the selected directory structure from different data sources. These data include financial data, market data, product sales, etc. Based on the selected structured data, users can use the industry dashboard tool to generate analytical charts. Analytical charts can include various types, such as bar charts, line charts, pie charts, etc., to show data performance in different dimensions. The purpose of these charts is to help users visualize data for further analysis and decision-making. The system recommends the most appropriate chart type based on the characteristics of the data. For example, for time series data, the system automatically recommends a line chart, while for categorical data, the system recommends a bar chart or a pie chart.

[0112] Furthermore, after creating the analysis charts, users can use the industry dashboard tool to arrange these charts in order to form a complete dashboard analysis framework. The arrangement order of the analysis charts is usually set according to the logical structure of the analysis. For example, some charts can be arranged in chronological order, while other charts are arranged by dimensions such as product category and region. Users can arrange charts of different dimensions in the industry dashboard in sequence. For example, the "Smartphone Market Share" chart is displayed first, followed by the "Sales Comparison of Different Brands" chart, and finally the "Annual Sales Growth Trend" chart. With this arrangement, users can clearly understand the overview of the entire market and the trend changes.

[0113] Furthermore, in order to ensure that the information in the dashboard analysis framework is always up to date, the system needs to regularly obtain the latest changes in the selected structured data. When the system obtains the latest structured data, the dashboard analysis framework will automatically update. For example, if the latest data of the "Smartphone Sales" chart changes, the system will automatically update the chart and readjust the data display in the industry dashboard. Users do not need to manually update the dashboard, the system will automatically perform all necessary update operations in the background. By updating the dashboard analysis framework in real time, users can always obtain accurate and up-to-date industry analysis data to provide immediate support for decision-making. The real-time update mechanism ensures efficiency and accuracy in the industry analysis process.

[0114] In one embodiment, after step S104, the method further includes:

[0115] Using the table editing function to query, retrieve and update the multi-level relational data structure to obtain a data calculation model;

[0116] Using the model management function to edit and save the data calculation model, and associating the industrial chain map with the data calculation model to obtain a dynamic industrial chain data model;

[0117] All data indicators in the dynamic industrial chain data model are updated through an indicator refresh mechanism.

[0118] In this embodiment, a table editing function is provided so that users can query and operate these data intuitively. The table editing function is based on the open source component LuckySheet and implements basic editing and calculation functions similar to Excel spreadsheets. Through the table editing function, users can perform the following operations on multi-level relational data:

[0119] Query: Users can use the table editing function to query data related to a specific industry link or product. For example, users can query data such as "sales volume", "market share" or "production cost" of the "smartphone" node;

[0120] Retrieving data: The table editing function allows users to retrieve relevant data from the database. Users can select different query conditions (such as time interval, company, product category, etc.) according to their needs to obtain specific data;

[0121] Update data: Users can update data in the table. For example, if the market share of a product changes, users can modify the data directly in the table and save the update.

[0122] By querying, retrieving and updating multi-level relational data, users can obtain data calculation models. The data calculation model is generated based on the industrial chain map, which includes the quantitative relationship and dynamic interaction between various industrial links.

[0123] Furthermore, in order to ensure the flexibility and operability of the data calculation model, the present invention provides a model management function. This function supports users to edit, update and save the generated data calculation model so that it can be called and modified multiple times in the subsequent analysis process. Specific functions include:

[0124] Edit model: Users can modify the data calculation model as needed. For example, users can add new analysis dimensions, such as adding a new "R&D investment" indicator, or modify the existing calculation formula;

[0125] Save model: After the model editing is completed, the user can save the model to the system for later use. The saved model can be associated with the data in the industrial chain map to ensure the consistency between the data and the model.

[0126] After editing and saving the data calculation model, the user can associate the model with the data in the industrial chain map. The association process includes:

[0127] Select associated nodes: Users can select one or more nodes in the industry chain map (such as the "smartphone" node, company node, etc.) and match them with the corresponding data fields in the data calculation model. In this way, the specific data in the industry chain map can be used as the input of the calculation model.

[0128] Establish dynamic data links: By establishing associations, each node in the industry chain map will automatically update the calculation results in the measurement model according to its data changes. For example, changes in smartphone sales will automatically affect the financial data model associated with it, thereby dynamically updating relevant data such as profit forecasts.

[0129] By associating the industrial chain map with the data calculation model, users can obtain a dynamic industrial chain data model. This dynamic industrial chain data model can not only perform real-time calculations based on changes in each link in the industrial chain, but also automatically adjust and optimize the analysis results when necessary.

[0130] Furthermore, in order to ensure that the data in the dynamic industrial chain data model is up to date, the system provides an indicator refresh mechanism, which automatically obtains and updates the data of each node in the industrial chain map on a regular basis to ensure that the data used in the model is real-time, thereby improving the accuracy and reliability of the model. The execution process of the indicator refresh mechanism is as follows:

[0131] Regularly obtain the latest industry data, company data, etc. through connections with external data sources or internal databases;

[0132] After obtaining the latest industry data, fill these data into the dynamic industry chain data model and update related indicators;

[0133] Through the real-time indicator refresh mechanism, the dynamic industrial chain data model can timely reflect changes in the industry and market, and support users to obtain accurate and timely data support in the decision-making process. Users can view and analyze the latest industrial chain data and measurement results at any time to make quick adjustments and decisions.

[0134] Combination Figure 4 As shown, Figure 4 A schematic block diagram of a data construction device based on an industrial chain map provided in an embodiment of the present invention, wherein data construction 400 based on the industrial chain map includes:

[0135] A data import unit 401 is used to import structured data and unstructured data from multiple data sources into a data warehouse respectively;

[0136] A data labeling unit 402 is used to perform standardized labeling on the structured data based on a predetermined labeling rule, and to perform standardized labeling on the unstructured data using a natural language processing model to obtain an industry label;

[0137] The data mounting unit 403 is used to construct an industrial chain map and mount the industrial label to the industrial chain map;

[0138] The data association unit 404 is used to perform data association on the industry tags based on the industry chain graph to obtain a multi-level relational data structure.

[0139] In this embodiment, the data import unit 401 imports structured data and unstructured data from multiple data sources into the data warehouse respectively; the data labeling unit 402 performs standardized labeling on the structured data based on predetermined labeling rules, and uses a natural language processing model to perform standardized labeling on the unstructured data to obtain industry labels; the data mounting unit 403 constructs an industrial chain map, and mounts the industry labels to the industrial chain map; the data association unit 404 performs data association on the industry labels based on the industrial chain map to obtain a multi-level relational data structure.

[0140] In one embodiment, the data importing unit 401 includes:

[0141] A data cleaning unit, used to extract the structured data and unstructured data from multiple data sources through an ETL tool, and perform data cleaning;

[0142] The import warehouse unit is used to import the cleaned data into different tables of the data warehouse by using database connection technology to obtain basic data.

[0143] In one embodiment, the data marking unit 402 includes:

[0144] A label classification unit, used to define the type label of the data set; wherein the type label includes sentiment type, company type, text type and event type;

[0145] A model training unit, used to train the natural language processing model using the type label, and optimize the accuracy of the natural language processing model by a hyperparameter tuning method to obtain a trained natural language processing model;

[0146] A label marking unit is used to use the trained natural language processing model to perform standardized labeling on the unstructured data to obtain the industry label.

[0147] In one embodiment, the data mounting unit 403 includes:

[0148] The product classification unit is used to classify the imported products based on the RoBERTa classification algorithm, so as to classify the products of different categories into corresponding categories according to the predefined industry standards and obtain industry data;

[0149] A data mapping unit, used for performing standardized mapping on the industry data to adjust the uniformity between different expressions of the same category of products and obtain standardized product data;

[0150] A graph mapping unit is used to edit and generate upstream and downstream supply relationships between products based on the standardized product data and industry data using knowledge graph technology to obtain the industrial chain graph.

[0151] In one embodiment, the data association unit 404 includes:

[0152] A graph adjustment unit, used to edit, adjust the format and optimize the layout of the industry label through a graphical interface of the industry chain graph;

[0153] An information association unit, used to automatically associate the nodes in the industry tag according to the attribute editing information input by the user;

[0154] The graphic connection unit is used to customize the upstream and downstream relationships between the nodes in the industry tag through a graphic connection function.

[0155] In one embodiment, after the data association unit 404, the following steps are included:

[0156] The permission acquisition unit is used to select the directory structure and configure permissions based on the preset industry dashboard building function according to the needs of industry analysis;

[0157] A chart creation unit, configured to select the structured data through a data browser interface based on the directory structure, and create an analysis chart using the structured data;

[0158] A framework construction unit, used for arranging the analysis charts in order to the industry dashboard to obtain a dashboard analysis framework;

[0159] The indicator updating unit is used to obtain the latest indicator changes of the structured data and update the dashboard analysis framework in real time.

[0160] In one embodiment, after the data association unit 404, the method further includes:

[0161] A model calculation unit, used to query, retrieve and update the multi-level relational data structure using a table editing function to obtain a data calculation model;

[0162] An industry association unit, used to edit and save the data calculation model using a model management function, and associate the industry chain map with the data calculation model to obtain a dynamic industry chain data model;

[0163] The model updating unit is used to update all data indicators in the dynamic industrial chain data model through an indicator refreshing mechanism.

[0164] Since the embodiments of the apparatus part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the apparatus part, which will not be repeated here.

[0165] The embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed, the steps provided in the above embodiment can be implemented. The storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0166] The embodiment of the present invention also provides a computer device, which may include a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps provided in the above embodiment may be implemented. Of course, the computer device may also include various network interfaces, power supplies and other components.

[0167] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of this application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of this application.

[0168] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. A data construction method based on an industrial chain graph, characterized in that: include: Import structured data and unstructured data from multiple data sources into the data warehouse respectively; The structured data is standardized and labeled based on a predetermined labeling rule, and the unstructured data is standardized and labeled using a natural language processing model to obtain an industry label; Construct an industrial chain map, and attach the industrial label to the industrial chain map; The industry tags are data-associated based on the industry chain graph to obtain a multi-level relational data structure.

2. The data construction method based on the industrial chain map according to claim 1 is characterized in that: The step of standardizing and marking the structured data based on a predetermined marking rule and standardizing and marking the unstructured data using a natural language processing model to obtain an industry label includes: Define the type labels of the data set; wherein the type labels include sentiment type, company type, text type and event type; Using the type label to train the natural language processing model, and optimizing the accuracy of the natural language processing model by a hyperparameter tuning method to obtain a trained natural language processing model; The trained natural language processing model is used to perform standardized labeling on the unstructured data to obtain the industry label.

3. The data construction method based on the industrial chain map according to claim 1 is characterized in that: The construction of the industrial chain map includes: Classify the imported products based on the RoBERTa classification algorithm to classify products of different categories into corresponding categories according to predefined industry standards and obtain industry data; Performing standardized mapping on the industry data to adjust the uniformity between different representations of the same category of products to obtain standardized product data; Based on the standardized product data and industry data, the knowledge graph technology is used to edit and generate the upstream and downstream supply relationships between products to obtain the industrial chain map.

4. The data construction method based on the industrial chain map according to claim 1 is characterized in that: The step of importing structured data and unstructured data from multiple data sources into the data warehouse includes: Extract the structured data and unstructured data from multiple data sources through ETL tools, and perform data cleaning; The cleaned data are imported into different tables of the data warehouse using database connection technology to obtain basic data.

5. The data construction method based on the industrial chain map according to claim 1 is characterized in that: The data association of the industry tags based on the industry chain graph is performed to obtain a multi-level relational data structure, including: Editing, formatting and optimizing the layout of the industry labels through the graphical interface of the industry chain map; Automatically associating nodes in the industry tag according to the attribute editing information input by the user; The upstream and downstream relationships between the nodes in the industry tags are customized through the graphic connection function.

6. The data construction method based on the industrial chain map according to claim 1 is characterized in that: After the data of the industry tags are associated based on the industry chain graph to obtain a multi-level relationship data structure, the method includes: Based on the preset industry dashboard building function, select the directory structure and configure permissions according to the needs of industry analysis; Selecting the structured data through a data browser interface based on the directory structure, and creating an analysis chart using the structured data; Arrange the analysis charts in order on the industry dashboard to obtain a dashboard analysis framework; The latest indicator changes of the structured data are obtained, and the dashboard analysis framework is updated in real time.

7. The data construction method based on the industrial chain map according to claim 1 is characterized in that: After the data of the industry tags are associated based on the industry chain graph to obtain a multi-level relationship data structure, the method further includes: Using the table editing function to query, retrieve and update the multi-level relational data structure to obtain a data calculation model; Using the model management function to edit and save the data calculation model, and associating the industrial chain map with the data calculation model to obtain a dynamic industrial chain data model; All data indicators in the dynamic industrial chain data model are updated through an indicator refresh mechanism.

8. A data construction device based on an industrial chain graph, characterized in that: include: A data import unit, used to import structured data and unstructured data from multiple data sources into the data warehouse respectively; A data labeling unit, used to perform standardized labeling on the structured data based on a predetermined labeling rule, and to perform standardized labeling on the unstructured data using a natural language processing model to obtain an industry label; A data mounting unit, used to construct an industrial chain map and mount the industrial label to the industrial chain map; The data association unit is used to perform data association on the industry tags based on the industry chain map to obtain a multi-level relational data structure.

9. A computer device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the data construction method based on the industrial chain map as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the data construction method based on the industrial chain map as described in any one of claims 1 to 7 is implemented.

Images