Standardized supervision method for industry big data product construction project

Abstract

The invention provides a standardized supervision method for an industry big data product construction project. The method comprises the following steps: acquiring an industry big data product construction project data packet, project preliminary association map elements and a product construction project document; based on the project preliminary association map elements and the product construction project document, constructing a project preliminary association map; constructing a simulation model based on the project preliminary association map, and quantifying the conduction weight of the indexes among the nodes by adopting the simulation model; forming a project complete map based on the conduction weights of the indexes among the nodes and the preliminary association map; analyzing the industry big data product construction project data packet to obtain operation data of each node corresponding to the map; determining an abnormal node range and an abnormal table cause based on the project complete map and the operation data of each node; and outputting the problem work order to the target terminal based on the abnormal node range and the abnormal table cause. By implementing the invention, the comprehensiveness of supervision can be improved.

Description

Technical Field

[0001] This invention belongs to the field of project supervision technology, specifically relating to a standardized supervision method for the construction of big data products in the industry. Background Technology

[0002] With the rapid development of the digital economy, big data technology has been deeply integrated into various industries, and the scale and complexity of big data product construction projects in these industries continue to rise. These projects involve multiple core stages such as data collection, cleaning, storage, analysis, and application. There are close data flow relationships between these stages, and each stage must meet specific business, performance, and compliance constraints. The quality of project construction directly determines the usability, accuracy, and security of big data products.

[0003] However, the supervision of current big data product development projects in the industry still faces many pain points, and traditional supervision methods are difficult to adapt to the complexity and dynamic needs of projects. Existing supervision relies heavily on manual verification of project documents and statistical data indicators, lacking a systematic understanding of the relationships between different stages of the project. It is impossible to build a global view of data flow between stages, resulting in one-sided supervision and difficulty in accurately locating hidden problems in the data flow process. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a standardized supervision method for industry big data product construction projects, so as to meet the comprehensive supervision requirements of industry big data product construction projects.

[0005] To achieve the above objectives, the present invention provides the following technical solution: According to a first aspect, the present invention provides a standardized supervision method for industry big data product construction projects, comprising: acquiring industry big data product construction project data packages, preliminary project correlation graph elements, and product construction project documents; constructing a preliminary project correlation graph based on the preliminary project correlation graph elements and the product construction project documents; constructing a simulation model based on the preliminary project correlation graph, and using the simulation model to quantify the transmission weights of indicators between various nodes; constructing a complete project correlation graph based on the transmission weights of indicators between various nodes and the preliminary correlation graph; parsing the industry big data product construction project data packages to obtain the operational data of each node corresponding to the graph; determining the scope of abnormal nodes and the causes of abnormalities based on the complete project correlation graph and the operational data of each node; and outputting a problem work order to the target terminal based on the scope of abnormal nodes and the causes of abnormalities.

[0006] According to a second aspect, the present invention provides a standardized monitoring device for industry big data product construction projects, comprising: a data acquisition module for acquiring industry big data product construction project data packages, preliminary correlation graph elements, and product construction project documents; a preliminary graph construction module for constructing a preliminary correlation graph based on the preliminary correlation graph elements and product construction project documents; a weight determination module for constructing a simulation model based on the preliminary correlation graph and using the simulation model to quantify the transmission weights of indicators between nodes; a complete graph determination module for constructing a complete graph based on the transmission weights of indicators between nodes and the preliminary correlation graph; a parsing module for parsing the industry big data product construction project data packages to obtain the operational data of each node corresponding to the graph; an anomaly determination module for determining the scope of abnormal nodes and the causes of anomalies based on the complete graph and the operational data of each node; and a communication module for outputting problem work orders to a target terminal based on the scope of abnormal nodes and the causes of anomalies.

[0007] According to a third aspect, an embodiment of the present invention provides an electronic device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the steps of the standardized supervision method for an industry big data product construction project described in the first aspect or any embodiment of the first aspect.

[0008] According to a fourth aspect, embodiments of the present invention provide a computer storage medium storing computer instructions that, when executed by a processor, implement the steps of a standardized supervision method for an industry big data product construction project as described in the first aspect or any embodiment of the first aspect.

[0009] This invention provides a standardized supervision method for industry big data product construction projects. From acquiring data and graph elements, to constructing a preliminary correlation graph, building a simulation model to quantify the weight of indicator transmission between nodes, to forming a complete graph and analyzing operational data, the method ultimately determines the scope and causes of abnormal nodes and outputs problem work orders. This breaks away from the limitations of traditional manual supervision by systematically organizing the relationships between various stages of the project through graphing and modeling methods, constructing a global view of data flow transmission, and accurately locating hidden problems. Simultaneously, by leveraging quantified indicator transmission weights, the method enhances the comprehensiveness of anomaly detection.

[0010] Other advantages, objectives, and features of the invention will be set forth in the following description and will be apparent to those skilled in the art in some respects, or may be learned by practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0011] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the following figures are provided for illustration: Figure 1 This is a flowchart illustrating a specific example of a standardized supervision method for an industry big data product construction project according to the present invention. Figure 2 This is a schematic diagram of a module structure of a standardized monitoring device for an industry big data product construction project according to the present invention; Figure 3 This is a schematic block diagram of a specific example of an electronic device in an embodiment of the present invention. Detailed Implementation

[0012] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0013] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can also refer to the internal connection of two components; and they can refer to a wireless connection or a wired connection. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0014] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0015] This invention provides a standardized supervision method for industry big data product construction projects, such as... Figure 1 As shown, it includes: S101, Obtain the data package of the industry big data product construction project, the preliminary correlation map elements of the project, and the product construction project documents; S102, Construct a preliminary project association map based on the elements of the preliminary project association map and the product construction project documents; S103, Based on the preliminary correlation map of the project, a simulation model is constructed, and the simulation model is used to quantify the transmission weight of indicators between various nodes; S104, based on the transmission weights of indicators between various nodes and the preliminary correlation map, constitutes the complete map of the project; S105, parse the data package of the industry big data product construction project to obtain the operation data of each node in the graph; S106, Based on the complete project map and the operational data of each node, determine the scope of abnormal nodes and the causes of abnormalities; S107, based on the range of abnormal nodes and the cause of the abnormality, outputs a problem work order to the target terminal.

[0016] For example, the supervision method proposed in this embodiment can be an algorithm applied within an integration platform. The integration platform includes a human-computer interaction interface (HCI) for receiving user input. The HCI can be a display interface, which includes an input interface for the data required by the platform. The industry big data product construction project data package and product construction project documents are imported into the HCI by the user. The industry big data product construction project data package contains actual operational data for each stage of the project, and the product construction project documents include requirements specifications, project technical design documents, etc. The preliminary project association graph elements include process stage entities and feature entities. In addition, it may include the connection relationships between various entities. Process stage entities represent different processes in the project, and feature entities include data source entities, algorithm model entities, and service interface entities.

[0017] Element generation supports two methods to meet the needs of different scenarios. First, template-based generation: the platform includes standard process entity modules and feature entity modules for big data product construction projects across various industries. Users can select the required modules through drag-and-drop operations, establish relationships between modules through the connection function, and supplement module attributes through the annotation function. Second, custom generation: for personalized project scenarios, users can create custom process link entities or feature entities through the new module function, setting module attribute parameters and association rules to improve platform adaptability. After element generation, it can be saved as a template for reuse in similar projects.

[0018] Based on the preliminary project association graph elements and product construction project documents, a preliminary project association graph is constructed, including: performing natural language parsing on the product construction project documents based on the preliminary project association graph elements to extract the mapping relationships between corresponding process link entities and feature entities, as well as the indicator constraints between process link entities and feature entities; constructing a preliminary association graph based on the mapping relationships between process link entities and feature entities, as well as the indicator constraints between process link entities and feature entities, the preliminary association graph includes an upper-level graph and a lower-level graph. The upper-level graph uses process link entities as the main nodes, with the data flow direction and indicator constraints of the main nodes as edges. The lower-level graph uses feature entities as child nodes, with the data flow direction and indicator constraints of the child nodes as edges. There are mapping relationships between the main nodes and multiple feature entity child nodes of the corresponding process links.

[0019] Specifically, the system uses process stage entities as the core and maps them to feature entities. For example, the data acquisition stage entity can be associated with the data source entity and the service interface entity, while the model training stage entity can be associated with the algorithm model entity and the data source entity. The graph is presented in a visual format and actually consists of two layers: the upper layer represents the stage connection relationships, and the lower layer represents the feature entity connection relationships, with a mapping relationship between the two layers.

[0020] Next, based on existing entity names, natural language processing (NLP) technology is used to obtain the requirement specifications and technical standards from the product project documentation. Indicator constraints are then set for each edge of the graph, resulting in a preliminary project association graph. Constraint types are divided into quantitative and qualitative constraints. Quantitative constraints include specific numerical thresholds, such as data transmission timeliness of less than 5 seconds, data accuracy greater than 99.5%, model training time less than 2 hours, and data timeliness within 2 days. Qualitative constraints include rule-based requirements, such as encrypted data transmission, permission verification for API calls, model training adaptability to business scenarios, and data comprehensiveness. Data comprehensiveness can be determined by predefined data categories, ensuring that all required data types are included. Indicator constraints support user-defined configurations, and the platform also provides industry-standard constraint templates that users can modify as needed.

[0021] Based on the established preliminary project relationship graph, a simulation model that closely reflects the actual operational logic of the project is constructed. First, the nodes and relationships in the preliminary relationship graph are refined, clarifying the core attributes and operational rules of each element. For process entities, their input / output standards, execution duration range, dependent preconditions, and resource consumption standards are broken down. For feature entities, their core parameters are defined, such as the update frequency and data format of data source entities, the iteration cycle and hyperparameter range of algorithm model entities, and the concurrency limit and retry mechanism for service interface entities. For related edges, their data transmission rules, timing logic, and exception triggering conditions are refined, providing accurate foundational elements for model construction. Then, the actual business processes are bound to the graph elements. This process can be accomplished through integration with the platform or external simulation tools, such as Arena and PlantSimulation.

[0022] Then, by simulating changes in upstream indicators, the response changes in downstream related indicators are observed, and the transmission weight values ​​of each related edge are calculated. This includes: constructing a simulation model based on the preliminary correlation map of the project; using the simulation model, perturbing any indicator value in the upstream link through Monte Carlo simulation, and recording the statistical changes in the indicator values ​​of the downstream link; and using regression analysis based on the statistical changes in the indicator values ​​of the downstream link to calculate the contribution ratio of changes in upstream indicators to changes in downstream indicators, which is used as the transmission weight.

[0023] Specifically, firstly, a simulation model is constructed based on the preliminary association graph of the project. The entities in the process flow of the graph are transformed into simulation activity modules with input / output queues, processing logic, and resource usage rules; feature entities are transformed into simulation resource / parameter modules; and the edges connecting entities are transformed into simulation transmission logic containing data transmission rules, timing dependencies, and anomaly triggering conditions. Simultaneously, the input / output index sets and transformation functions of each node are defined, simulation engines such as Arena and AnyLogic are selected, discrete event or time step progression mechanisms are configured, and a fixed random seed is set to ensure the reproducibility of the results.

[0024] Secondly, Monte Carlo simulations were performed to generate sample data. Core indicators of the upstream nodes to be analyzed were selected, and normal baseline values ​​and perturbation ranges were set. Following the principles of single perturbation, multiple repetitions (more than 1000 repetitions at each level), and randomization of the experimental order, simulation experiments were performed group by group according to different perturbation levels. The upstream indicator values, corresponding downstream indicator values, and simulation timestamps of each group of experiments were recorded to form a standardized experimental dataset.

[0025] Next, the transmission weights are calculated through regression analysis. The simulated data is first cleaned and Z-score standardized, then divided into training and test sets in a 7:3 ratio. Models are selected based on the upstream and downstream indicator relationships, including multiple linear regression and nonlinear regression models. The transmission weights are calculated using standardized regression coefficients, variance contribution decomposition, or elasticity coefficients, and the results are normalized to the 0-1 interval. The stability of the weights is verified through K-fold cross-validation and residual analysis, and adjustments are made based on economic significance testing.

[0026] Next, the data package of the industry big data product construction project is analyzed to obtain the operational data of each node in the graph. This is the actual operational data of the industry big data product. Through the operational data, the achievement of the node's indicators can be determined.

[0027] Based on this, the scope of abnormal nodes can be further determined based on the complete project graph and the operational data of each node. This includes: screening preliminary abnormal nodes based on the operational data of each node and the indicator constraints between nodes in the upper-level graph of the complete project graph. Preliminary abnormal nodes are nodes in the upper-level graph whose operational data does not meet the indicator constraints; when the downstream node of the preliminary abnormal node is also a preliminary abnormal node, the abnormal impact value of the first preliminary abnormal node on its downstream related nodes is calculated based on the indicator transmission weight in the first preliminary abnormal node. The downstream related nodes are nodes in the upper-level graph; based on the abnormal impact value and the operational data of the downstream related nodes, the abnormal type of the downstream related nodes is determined. The abnormal types include spontaneous abnormalities and linked abnormalities.

[0028] When a linkage anomaly occurs, the child node of the lower-level graph corresponding to the first preliminary anomaly node is determined through the mapping relationship; the range of the child node anomaly is determined based on the running data and indicator transmission weights of the child node of the lower-level graph corresponding to the first preliminary anomaly node; and the range of the anomaly node is determined based on the range of the child node anomaly.

[0029] When the anomaly is spontaneous, the child nodes of the lower-level graph corresponding to the first preliminary anomaly node and the child nodes of the lower-level graph corresponding to the downstream associated nodes are determined through the mapping relationship. Based on the child nodes of the lower-level graph corresponding to the first preliminary anomaly node and the indicator transmission weight, the range of anomaly nodes of the child nodes of the first preliminary anomaly node is determined. Based on the child nodes of the lower-level graph corresponding to the downstream associated nodes and the indicator transmission weight, the range of anomaly nodes of the child nodes of the downstream associated nodes is determined. Based on the range of anomaly nodes of the child nodes of the first preliminary anomaly node and the range of anomaly nodes of the child nodes of the downstream associated nodes, the range of anomaly nodes is determined.

[0030] Specifically, each upper-level node has quantifiable operational metrics, such as 99.2% data accuracy for node A, 150ms interface response latency for node B, and 95% data coverage for node C. Each edge connecting an upper-level node has a corresponding explicit metric constraint rule, such as accuracy > 99.5% for the incoming edge of node A, response latency < 100ms for the incoming edge of node B, and coverage ≥ 98% for the incoming edge of node C.

[0031] For each upper-level node, firstly, find the constraints of all associated edges of the current node; then, compare the actual index value of the node with the corresponding constraint. If the actual value meets the constraint, such as node E having an actual accuracy of 99.6% and the constraint being >99.5%, then the node is determined to have no initial anomaly; if the actual value does not meet the constraint, such as node A having an actual accuracy of 99.2% < constraint 99.5%, then the node is marked as a preliminary anomaly node. After completing the above comparison for all upper-level nodes, summarize all marked nodes to form a preliminary anomaly node set.

[0032] Next, for each initial abnormal node, its abnormal deviation value is calculated. That is, the difference between the constraint threshold and the actual value: When the constraint is greater than the sign (e.g., accuracy > 99.5%): =Constraint threshold - Actual value (e.g., ΔA = 99.5% - 99.2% = 0.3%). When the constraint is less than the sign (e.g., delay < 100ms): = Actual value - constraint threshold (e.g., ΔB = 150ms - 100ms = 50ms); When the constraint is equal to the sign: =|Actual value - Constraint threshold| (Example: If the constraint is accuracy == 99%, and the actual value is 98.5%, then Δ = 0.5%).

[0033] The initial set of abnormal nodes may include upstream and downstream nodes. This step is to determine whether the abnormality of the downstream node is caused by the upstream node (linkage) or by its own cause (spontaneous). The specific steps are as follows: First, construct the propagation chain of abnormal nodes. Based on the node connection relationship of the upper-level graph, clarify the upstream and downstream relationship of abnormal nodes. For example, in the preliminary abnormal set {A,B,C}, A→B→C is a propagation chain, where A is upstream of B and B is upstream of C. Arrange the nodes in the order of upstream→downstream.

[0034] For each pair of upstream and downstream anomalous nodes, calculate the expected impact of the upstream node on the downstream node. Specifically, the input data is the anomalous deviation value of the upstream node. The weight W of the indicator transmission between upstream and downstream nodes; Expected impact = ×W.

[0035] A pre-set judgment threshold θ (determined by business experience, typically 0.7~0.8) is used to determine the anomaly type by comparing the expected impact with the actual deviation value of the downstream node. Step 1: Calculate the actual abnormal deviation value of the downstream node. ; Step 2: Calculate the coefficient of determination = expected impact / ; Step 3: Compare the decision coefficient with the threshold θ: If the determination coefficient (e.g., 0.96>0.7): Determine that the downstream node is experiencing a linkage anomaly (i.e., the anomaly of B is mainly caused by A). If the determination coefficient (As expected, the impact is 0.2%, Δdownstream = 0.8%, coefficient = 0.25) 0.7): Determine that the downstream node is spontaneously abnormal (i.e., the abnormality of B is caused by its own independent reasons and is unrelated to A).

[0036] The following are specific examples: Node A (upstream): A = 0.3%, W = 0.8 for A → B, expected impact = 0.3% × 0.8 = 0.24%; Node B (downstream): B=0.25%, judgment coefficient=0.24 / 0.25=0.96>0.7→B is an abnormal linkage; The W value for the transition from node B to C is 0.9, and the expected impact is 0.25% × 0.9 = 0.225%. Node C: C=0.5%, coefficient of determination = 0.225 / 0.5 = 0.45 < 0.7 → C is a spontaneous anomaly.

[0037] Finally, based on the anomaly type (linked / spontaneous), the final range of anomaly nodes to be investigated is determined as follows: Scenario 1: Interconnection anomaly (downstream anomaly caused by upstream) The root cause of all linked anomalies lies in the first abnormal node in the propagation chain (i.e., the upstream abnormal node). The anomalies in downstream nodes are a chain reaction and do not need to be investigated separately. It is only necessary to trace the root cause of the first node.

[0038] Execution steps: Find the first abnormal node in the propagation chain, i.e., the node without an upstream abnormal node. For example, in A→B→C, A is the first node, and B / C are linked. Trace down to the feature entity child node corresponding to the first node in the lower layer graph (e.g., the upper layer node A corresponds to the lower layer data source X, interface Y, and computation task Z). Check the running data of these lower layer child nodes and use the propagation weights between lower layer child nodes to further locate the root cause: for example, in the lower layer child nodes, the update delay of data source X causes the accuracy of computation task Z to decrease, which eventually triggers the abnormality of upper layer A. Output the final abnormal node range: the first abnormal node (A) + the set of its mapped lower layer abnormal child nodes {data source X, computation task Z}.

[0039] Scenario 2: Spontaneous anomaly (node ​​anomaly is an independent cause) Each spontaneously anomalous node has an independent root cause, and the root cause needs to be traced for each spontaneously anomalous node separately.

[0040] Execution steps: Identify all spontaneously occurring abnormal nodes. For example, in the initial abnormal set, A is the first abnormal node, B is a linked abnormal node, C is spontaneous, and D is spontaneous. For each spontaneously occurring abnormal node, find its corresponding set of feature entity sub-nodes in the lower-level graph. For example, spontaneous node C corresponds to the lower-level {storage cluster M, synchronization task N}, and spontaneous node D corresponds to the lower-level {caching service P, log collection Q}. In the lower-level sub-graph corresponding to each spontaneous node, independently analyze the abnormal propagation. For example, analyze the lower-level nodes of C: high disk I / O of storage cluster M causes synchronization task N to fail, ultimately triggering the abnormality of C. Analyze the lower-level nodes of D: low hit rate of cache service P causes D's response latency to exceed the standard. Output the final range of abnormal nodes: the union of the lower-level abnormal sub-node sets corresponding to all spontaneously occurring abnormal nodes.

[0041] By using the above method, we first identify potential anomalies in the upper-level process, avoiding indiscriminate investigation of all nodes, narrowing the scope of the problem from the source, and reducing ineffective operations. At the same time, by quantifying the impact of upstream on downstream, we can determine linkage anomalies, reduce redundant investigation steps, improve anomaly investigation efficiency, and reduce algorithm overhead.

[0042] Finally, the identified cause of the anomaly is used as an element in a problem ticket, a problem ticket is generated, and sent to the target terminal corresponding to the anomaly node. It should be noted that the cause of the anomaly has already been determined in the steps above for defining the scope of the anomaly node, such as the update delay of data source X.

[0043] This invention provides a standardized supervision method for industry big data product construction projects. From acquiring data and graph elements, to constructing a preliminary correlation graph, building a simulation model to quantify the weight of indicator transmission between nodes, to forming a complete graph and analyzing operational data, the method ultimately determines the scope and causes of abnormal nodes and outputs problem work orders. This breaks away from the limitations of traditional manual supervision by systematically organizing the relationships between various stages of the project through graphing and modeling methods, constructing a global view of data flow transmission, and accurately locating hidden problems. Simultaneously, by leveraging quantified indicator transmission weights, the method enhances the comprehensiveness of anomaly detection.

[0044] As an optional implementation method, based on the range of abnormal nodes and the cause of the abnormality, a problem work order is output to the target terminal, including: determining the target terminals bound to the nodes within the range of abnormal nodes; generating multiple problem work orders for the cause of the abnormality of each abnormal node within the range of abnormal nodes; and outputting the problem work orders to the corresponding target terminals respectively.

[0045] For example, the system first retrieves the pre-stored node-terminal binding relationship database on the platform to obtain basic information about the target terminal, including terminal number, responsible party, receiving address / port, and terminal type. Using the defined range of abnormal nodes as the filtering criteria, the system accurately retrieves and batches target terminal information corresponding to all abnormal nodes within that range from the relationship database by node identifier. Then, for each abnormal node within the range of abnormal nodes, based on its actual cause of the abnormality, multiple problem work orders are generated according to a preset work order template. These work orders include information about the abnormal node, a description of the cause of the abnormality, processing requirements, response time limits, and associated target terminal information. Finally, each generated problem work order is directed to the corresponding target terminal according to preset rules such as the receiving address / port of the target terminal, achieving precise matching and delivery of problem work orders to target terminals.

[0046] This invention provides a standardized supervision method for industry big data product construction projects. By binding nodes and terminals, it enables precise and targeted transmission of problem work orders, avoiding response delays caused by incorrect or missed work orders. Targeted output ensures that responsible terminals can receive the corresponding work orders in a timely manner, quickly initiate the exception handling process, and improve the timeliness of project problem response.

[0047] This embodiment provides a standardized monitoring device for industry big data product construction projects, such as... Figure 2 As shown, it includes: The data acquisition module 201 is used to acquire data packages for industry big data product construction projects, preliminary association map elements of the project, and product construction project documents. The preliminary association map elements of the project include nodes, which are the process links and feature entities of the project. The preliminary graph construction module 202 is used to construct a preliminary project association graph based on the preliminary project association graph elements and product construction project documents. The edges of the preliminary project association graph are set with indicator constraints. The weight determination module 203 is used to construct a simulation model based on the preliminary correlation map of the project, and to quantify the transmission weight of indicators between each node using the simulation model. The complete graph determination module 204 is used to construct the complete project graph based on the transmission weights of indicators between various nodes and the preliminary correlation graph; The parsing module 205 is used to parse the data package of the industry big data product construction project to obtain the running data of each node in the graph; The anomaly determination module 206 is used to determine the range of abnormal nodes and the causes of anomalies based on the complete project map and the operational data of each node. The communication module 207 is used to output a problem work order to the target terminal based on the range of abnormal nodes and the cause of the abnormality.

[0048] This application also provides an electronic device, such as... Figure 3 As shown, processor 501 and memory 502 are connected via a bus or other means.

[0049] Processor 501 can be a central processing unit (CPU). Processor 501 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips.

[0050] The memory 502, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the standardized supervision method for an industry big data product construction project in this embodiment of the invention. The processor executes various functional applications and data processing by running the non-transitory software programs, instructions, and modules stored in the memory.

[0051] Memory 502 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor, etc. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 502 may optionally include memory remotely located relative to the processor, which can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0052] The one or more modules are stored in the memory 502, and when executed by the processor 501, they perform actions such as... Figure 1 The embodiment shown illustrates a standardized supervision method for an industry big data product construction project.

[0053] For specific details regarding the aforementioned electronic devices, please refer to the relevant documentation. Figure 1 The relevant descriptions and effects in the illustrated embodiments are for understanding purposes only and will not be repeated here.

[0054] This embodiment also provides a computer storage medium storing computer-executable instructions that can execute a standardized supervision method for an industry big data product construction project in any of the above method embodiments. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium may also include combinations of the above types of memory.

[0055] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made to it in form and detail without departing from the scope defined by the claims of the present invention.

Claims

1. A standardized supervision method for industry big data product construction projects, characterized in that, include: Obtain industry big data product construction project data packages, preliminary project correlation map elements, and product construction project documents; Based on the elements of the preliminary project association map and the product construction project documents, a preliminary project association map is constructed. A simulation model is constructed based on the preliminary correlation map of the project. The simulation model is then used to quantify the transmission weight of indicators between various nodes. Based on the transmission weights of indicators between each node and the preliminary correlation graph, a complete project graph is constructed. Analyze the data package of the industry big data product construction project to obtain the operational data of each node in the graph; Based on the complete project map and the operational data of each node, the scope of abnormal nodes and the causes of the abnormalities were determined. Based on the range of abnormal nodes and the cause of the abnormality, a problem work order is output to the target terminal.

2. The standardized supervision method for industry big data product construction projects according to claim 1, characterized in that, The preliminary project association graph elements include nodes, which are project process entities and feature entities. Based on the preliminary project association graph elements and product construction project documents, a preliminary project association graph is constructed, including: Based on the preliminary association map elements of the project, natural language parsing is performed on the product construction project documents to extract the mapping relationship between corresponding process entities and feature entities, as well as the indicator constraints between process entities and feature entities. Based on the mapping relationship between process step entities and feature entities, as well as the indicator constraints between process step entities and feature entities, a preliminary association graph is constructed. The preliminary association graph includes an upper-level graph and a lower-level graph. The upper-level graph uses process step entities as the main nodes and the data flow direction and indicator constraints of the main nodes as edges. The lower-level graph uses feature entities as child nodes and the data flow direction and indicator constraints of the child nodes as edges. There is a mapping relationship between the main node and multiple feature entity child nodes of the corresponding process step.

3. The standardized supervision method for industry big data product construction projects according to claim 1, characterized in that, A simulation model is constructed based on the preliminary correlation map of the project. This model is then used to quantify the transmission weights of indicators between different stages, including: A simulation model was constructed based on the preliminary correlation map of the project. Using a simulation model, Monte Carlo simulation is employed to perturb any index value in the upstream process, and the statistical changes in the index value in the downstream process are recorded. Based on the statistical changes in downstream indicators, regression analysis is used to calculate the contribution ratio of upstream indicator changes to downstream indicator changes, which serves as the transmission weight.

4. The standardized supervision method for industry big data product construction projects according to claim 2, characterized in that, Based on the complete project map and the operational data of each node, the range of abnormal nodes was determined, including: Based on the operational data of each node and the indicator constraints between nodes in the upper-level graph of the complete project map, preliminary abnormal nodes are screened. Preliminary abnormal nodes are nodes in the upper-level graph whose operational data does not meet the indicator constraints. When the downstream node of the initial anomalous node is also an initial anomalous node, the anomalous impact value of the first initial anomalous node on its downstream associated nodes is calculated based on the indicator transmission weight in the first initial anomalous node. The downstream associated nodes are the nodes in the upper-level graph. Based on the abnormal impact value and the operational data of downstream related nodes, the abnormal type of downstream related nodes is determined. The abnormal types include spontaneous abnormalities and linkage abnormalities. When a linkage anomaly is detected, the child node of the lower-level graph corresponding to the first preliminary anomaly node is determined through the mapping relationship. Based on the operational data of the child nodes in the lower-level graph corresponding to the first preliminary abnormal node and the indicator transmission weights, the range of abnormal nodes in the child nodes is determined. The range of abnormal nodes is determined based on the range of abnormal child nodes.

5. The standardized supervision method for industry big data product construction projects according to claim 4, characterized in that, Also includes: When it is a spontaneous anomaly, the child nodes of the lower-level graph corresponding to the first initial anomaly node and the child nodes of the lower-level graph corresponding to the downstream associated nodes are determined through the mapping relationship. Based on the child nodes of the lower-level graph corresponding to the first preliminary anomaly node and the indicator transmission weight, the range of anomaly nodes of the child nodes of the first preliminary anomaly node is determined. Based on the child nodes of the lower-level graph corresponding to the downstream associated nodes and the indicator transmission weights, the range of abnormal nodes of the downstream associated node child nodes is determined. The range of abnormal nodes is determined based on the range of abnormal nodes of the first preliminary abnormal node's child nodes and the range of abnormal nodes of the downstream related node's child nodes.

6. The standardized supervision method for industry big data product construction projects according to claim 1, characterized in that, Based on the scope of the abnormal nodes and the cause of the abnormality, a problem ticket is output to the target terminal, including: Based on the range of abnormal nodes, determine the target terminals bound to the nodes within that range; For each abnormal node within the scope of abnormal nodes, generate multiple problem work orders based on the cause of the abnormality. Output the problem work orders to the corresponding target terminals.

7. A standardized supervision method for industry big data product construction projects according to any one of claims 1-6, characterized in that, The constraints include the comprehensiveness and timeliness of data collection.

8. A standardized monitoring device for industry big data product construction projects, characterized in that, include: The data acquisition module is used to acquire data packages for industry big data product construction projects, preliminary correlation map elements of the projects, and product construction project documents; The preliminary map construction module is used to construct a preliminary project association map based on the preliminary association map elements and product construction project documents. The weight determination module is used to build a simulation model based on the preliminary correlation map of the project, and to quantify the transmission weight of indicators between each node using the simulation model. The complete graph determination module is used to construct the complete project graph based on the transmission weights of indicators between various nodes and the preliminary correlation graph; The parsing module is used to parse the data packets of industry big data product construction projects to obtain the operational data of each node in the graph. The anomaly determination module is used to determine the range of abnormal nodes and the causes of anomalies based on the complete project map and the operational data of each node. The communication module is used to output problem work orders to the target terminal based on the range of abnormal nodes and the cause of the abnormality.

9. An electronic device, the device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor performs the steps of a standardized supervision method for an industry big data product construction project as described in any one of claims 1-7.

10. A computer storage medium storing computer instructions thereon, characterized in that, When executed by the processor, this instruction implements the steps of a standardized supervision method for an industry big data product construction project as described in any one of claims 1-7.

Images