A data call chain reconstruction and analysis system based on big data

Abstract

The present invention discloses a data call link restoration analysis system based on big data, including: a data acquisition engine, including a scheduled task acquisition unit and a flume real-time acquisition unit; a data analysis engine, for receiving data in kafka, and converting it into a set data structure, and then using rds audit log, sls audit log, and arms application stack information as analysis entrances, respectively, to perform full-link tracking process analysis; a data storage engine, for hierarchical storage of collected source data, cleaned and converted data, and data processed by a data analysis engine; a data display component, for obtaining data processed by a data analysis engine, and displaying full-link analysis results from three dimensions: data, microservices, and application systems. The present invention can solve the problem that current link tracking cannot obtain information such as the application to which the service belongs, what data in the database the service specifically calls, and the data flow link from the data center to the database.

Description

Technical Field

[0001] The present invention relates to the technical field of data link tracking, and in particular to a data call link restoration and analysis system based on big data. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] The rise of distributed architecture has promoted the development of some new technologies. Among them, link tracking technology has become an indispensable part of distributed architecture with its excellent performance in the field of APM. The current link tracking technology mainly realizes the dependency relationship between services, the interface call relationship between services, and the database information of service calls, but it cannot obtain information such as the application system to which the service belongs, what data the service specifically calls in the database, and the data flow link from the data center to the database.

[0004] With the gradual improvement of the data middle platform, the data usage process is no longer a simple database call, but through data integration to the data source, and then through Jdbc or data service mode for microservice call. In the existing technology, the detailed link of the data in the entire call process cannot be fully tracked, resulting in frequent data access risks. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes a data call link restoration analysis system based on big data, which realizes the full-link restoration analysis of data access by uniformly collecting data integration information, data source information, database audit information, APM application monitoring information, cloud resource set information, cloud ECS resource information, etc. in the data access process, data cleaning and conversion, data analysis based on big data, and displaying the analysis results.

[0006] In some embodiments, the following technical solutions are adopted:

[0007] A data call link restoration analysis system based on big data, comprising:

[0008] The data collection engine includes a scheduled task collection unit and a flume real-time collection unit. The scheduled task collection unit is used to collect data from different interfaces regularly for subsequent data flow link restoration analysis; the flume real-time collection unit is used to convert log data into json files and then collect and store them in kafka in real time through the flume component;

[0009] The data analysis engine is used to receive data from Kafka and convert it into a specified data structure. It then uses RDS audit logs, SLS audit logs, and Arms application stack information as analysis entry points to perform full-link tracking process analysis.

[0010] The data storage engine is used to store the collected source data, the cleaned and converted data, and the data processed by the data analysis engine in layers;

[0011] The data display component is used to obtain data processed by the data analysis engine and display the full-link analysis results from three dimensions: data, microservices, and application systems.

[0012] As an optional implementation, the data analysis engine includes:

[0013] Impala SQL analysis component, used to perform SQL analysis using RDS audit logs, SLS audit logs, and Arms application stack information as analysis entry points;

[0014] Spark streaming is a streaming processing component that receives data from Kafka and converts it into the data structure Dstream in spark streaming. The job submitted by spark streaming will perform multi-dimensional correlation analysis on the data.

[0015] As an optional implementation method, the full-link tracking process analysis is performed using the RDS audit log, SLS audit log, and ARMs application stack information as analysis entries, including:

[0016] The spark streaming component consumes the Kafka message queue, periodically obtains the latest offsets in each partition of each topic in Kafka, and then processes each batch according to the set maxRatePerPartition;

[0017] The spark streaming component uses the impala component to obtain intermediate connection data, and uses the Spark streaming component and the imapala script to perform association and SQL connection analysis;

[0018] The analyzed data is stored back in Kafka and two message topics are generated, one for storage and the other for display.

[0019] The spark streaming component consumes the storage information topic, analyzes the data flow link, and stores the analyzed data in HDFS or KUDU database;

[0020] The JAVA microservice periodically polls the consumption display information topic to store data link display information, which is pushed to the front-end micro application through websocket for real-time display.

[0021] As an optional implementation, the analysis includes the following three scenarios:

[0022] Analyze the complete link of data calls with the help of stack SQL information obtained through application monitoring;

[0023] Use RDS audit logs combined with monitoring information to analyze data and call the complete link;

[0024] Use sls audit logs to analyze the complete link of data service calls.

[0025] As an optional implementation, the complete link of analyzing data calls using stack SQL information acquired through application monitoring specifically includes:

[0026] Use the stack information of application monitoring to obtain SQL information and link identifiers, then analyze the table and field information of the call database by parsing SQL, and query the microservice IP and microservice unique identifier through the link bid;

[0027] The associated database IP and port are queried through the microservice unique identifier, and the cloud database instance information is matched to obtain the project space where the data source is located;

[0028] By connecting to data integration information and DataWorks data source information, you can obtain detailed information about the entire access link.

[0029] As an optional implementation, the complete link of calling the data by using the RDS audit log combined with the monitoring information analysis data specifically includes:

[0030] Analyze the microservice IP, SQL, and RDS instance information that access the data through RDS audit logs, analyze the table and field information of the database by parsing SQL, obtain the instance IP through the cloud database unique identifier, and determine the target library and task space ID by comparing the instance IP with the IP of the data source;

[0031] The microservice IP obtains the microservice name by connecting to the application monitoring information, and obtains the business system and department to which the microservice belongs by connecting to the cloud server instance information through the microservice IP;

[0032] By querying the data integration table through the unique identifier of the task space and comparing the target library and target table, the data middle platform project space and data middle platform table are obtained, and finally the full-link information of the application scenario in which the data integration is synchronized to the cloud database and the service is directly connected to the cloud database through JDBC is obtained.

[0033] As an optional implementation, the complete link of analyzing the data service call with the help of sls audit log is specifically as follows:

[0034] Obtain the microservice IP, SQL information, and cloud database unique identifier of the microservice accessing the data service through the SLS audit log. Analyze the table and field information of the target database by parsing SQL. Query the cloud database name and domain name through the cloud database unique identifier. Then obtain the cloud database IP by parsing the domain name. Connect to the external data source information and obtain the target database and task space unique identifier.

[0035] By comparing and querying the data integration table with the unique identifier of the external database, external table and task space, the middle platform project space and middle table can be obtained; the microservice IP can obtain the microservice name, the business system where the microservice is located, and the business department information by comparing the application monitoring information and the cloud server instance information;

[0036] Finally, the full-link information is analyzed, synchronized to the cloud database through data integration and published as a data service for calling application scenarios.

[0037] As an optional implementation, the data storage engine is divided into three layers: a source layer, a sharing layer, and an analysis layer. The source layer is used to store the collected source data, the sharing layer is used to periodically clean and convert the data in the source layer, and store the results in the sharing layer; the analysis layer is used to store data processed by the analysis engine.

[0038] As an optional implementation, the data for regularly cleaning the source layer is specifically:

[0039] Filter out the data that does not meet the requirements and hand over the filtering results to the business department to confirm whether it should be filtered out or revised by the business unit before extraction.

[0040] As an optional implementation, the data for regularly cleaning the source layer is specifically:

[0041] Unify the same type of data from different business systems, aggregate the source layer data according to the subject domain granularity of the shared layer, and calculate different data indicators in different business rules.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] (1) The present invention can solve the problem that the current link tracking cannot obtain information such as the application to which the service belongs, what data in the database the service specifically calls, and the data flow link from the data center to the database. Users can easily obtain full-link information in the data call process by calling the interface disclosed by the present invention in their own systems.

[0044] (2) The present invention can seamlessly connect the data in each access process to form a complete access link and realize real data full-link tracking. The scheduled collection engine can realize various forms of data collection by configuring the execution method and CRON expression on the front page and using the FEIGN component in the background to communicate between services; the real-time collection engine first lands the data into a json file, and then collects it in real time through the flume component, supports massive log collection, aggregation and transmission, and supports the collection of any stream event data; Kafka is used to realize data pulling and consumption, and the data is classified through TOPIC, supporting multiple producers and consumers, supporting the horizontal expansion of BROKER, solving the problem of large data volume and multiple types and inconsistent consumption time and mechanism; spark streaming streaming processing components are used for analysis to achieve quasi-real-time response, and the fault tolerance cost is low.

[0045] Other features and advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a structural diagram of a data call link restoration and analysis system based on big data in an embodiment of the present invention;

[0047] Figure 2 A schematic diagram of the process of tracking a full-link scenario by using application monitoring information in an embodiment of the present invention;

[0048] Figure 3 This is a schematic diagram of the process of tracing the full-link scenario by using RDS audit logs in an embodiment of the present invention;

[0049] Figure 4 The figure is a schematic diagram of the process of tracing the full-link scenario with the help of SLS audit logs in an embodiment of the present invention. DETAILED DESCRIPTION

[0050] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used in the present invention have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0051] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0052] Embodiment 1

[0053] In one or more embodiments, a data call link restoration analysis system based on big data is disclosed, combined with Figure 1 , including:

[0054] (1) Data collection engine, including a scheduled task collection unit and a Flume real-time collection unit; the scheduled task collection unit collects the interface data of the cloud platform resource set, data source, application monitoring, etc. through a Java scheduled task, and the collected data is directly stored in the big data platform for subsequent data flow link restoration analysis; the Flume real-time collection unit first converts the RDS database audit log, CDH database audit log, SLS data service call log and other log data into JSON files and then collects them in real time through the Flume component. The collected data is temporarily stored in Kafka, waiting for consumption by the analysis engine.

[0055] In this embodiment, the data access process includes terminals, users, business systems, microservices, interfaces, databases, and tables. To perform link analysis on the entire access process, it is necessary to collect audit data for each small process, including traffic logs, cloud resource set information, application monitoring information, data source information, database audit logs, etc.

[0056] The data collected in non-real time is implemented as a scheduled task by JAVA. By configuring the task name, task description, calling method, and CRON expression on the front page, the method, CRON expression, error strategy, and task name that need to be executed on a scheduled basis are passed to the background, and the required method is executed on a scheduled basis.

[0057] CRON expression is a string used to express the frequency and time of scheduled task execution. When the task is executed, the background of this module uses Spring Cloud Feign declarative communication between services, defines an interface that specifies the service to be called through the annotation @FeignClient(), and adds the startup class

[0058] @EnableFeignClients can turn on the Feign function, thereby calling the method of the cloud platform data acquisition module for scheduled execution and realizing the scheduled collection of interface data of cloud platform related components.

[0059] For data that needs to be collected in real time, first obtain the CDH audit log, RDS database audit log, SLS data service audit log and other data from the interface and temporarily store them in the server in the form of JSON files. Then, use the Flume component of the big data platform to collect and store them in real time in the data buffer Kafka and the data storage HDFS database.

[0060] (2) Data analysis engine, which receives data from Kafka and converts it into a specified data structure. It then uses the cloud database audit log, data service audit log, and application monitoring stack information as analysis entry points to perform full-link tracking process analysis.

[0061] In this embodiment, the data analysis engine is composed of two parts: the Impala SQL analysis component and the Spark Streaming stream processing component. Spark Streaming streams the data in Kafka. The first step is to receive the Kafka data and convert it into the data structure in Spark Streaming. The way to receive data is to use a receiver to receive the data in Kafka, using the Kafka high-level user API interface. For all receivers, the data received from Kafka will be stored in the Spark executor, and then the job submitted by Spark Streaming will perform multi-dimensional correlation analysis on the data. During the task processing process, the Impala SQL component is combined to perform SQL analysis, and the cloud database audit log, data service audit log, and application monitoring stack information are used as analysis entries to perform full-link tracking process analysis, which is written to Kafka after analysis. The non-real-time display data is directly stored in the warehouse, and the real-time display data is obtained by calling the Kafka interface through the microservice micro-application to display the data for the front-end.

[0062] (3) a data storage engine, which is used to hierarchically store the collected source data, the cleaned and converted data, and the data processed by the data analysis engine;

[0063] In this embodiment, the function of the data storage engine is to store the collected source data as big data, and divide the storage into three layers: source layer, sharing layer, and analysis layer. The source layer is used to store the collected source data, the sharing layer stores the cleaned and converted data, and the analysis layer mainly stores the data processed by the analysis engine.

[0064] In this embodiment, the data collected by the collection engine is stored in the source layer, and the original data is retained to facilitate the subsequent data tracing.

[0065] The data of the shared layer is mainly cleaned and converted by configuring scheduled tasks in the kettle component of the big data platform, and the results are stored in the shared layer. Among them, the task of data cleaning is to filter out those data that do not meet the requirements, and hand over the filtered results to the business department to confirm whether to filter them out or extract them after correction by the business unit. The data that does not meet the requirements mainly includes three categories: incomplete data, erroneous data and duplicate data. The task of data conversion is mainly to perform inconsistent data conversion, data granularity conversion and business rule calculation. Inconsistent data conversion unifies the same type of data from different business systems. For data granularity conversion, the source layer generally stores very detailed data, while the data of the shared layer is used for analysis, which does not require very detailed data. The source layer data is aggregated according to the subject domain granularity of the shared layer. For the calculation of business rules, different departments have different business rules and different data indicators. These indicators cannot be completed by simple addition and subtraction. These data indicators need to be calculated in the data transmission tool and stored in the shared layer for analysis.

[0066] The analysis layer mainly stores data processed by the analysis engine. The data processed by the analysis engine is connected to HDFS through Kafka and stored in the analysis layer of the big data platform for display or external data sharing.

[0067] (4) Data display component, which is used to obtain data processed by the data analysis engine and display the full-link analysis results from three dimensions: data, microservices, and application systems.

[0068] In this embodiment, the data display component mainly uses components such as vue, echarts, Antv-G6, etc. to display in micro-applications in the form of charts, topology diagrams, etc. The results after the full-link analysis are obtained by obtaining the results processed by the analysis engine in kafka through the microservice interface, and the display component displays the full-link analysis results in three dimensions: data, microservices, and application systems, to achieve visualization of data flow in the data center.

[0069] The data display component of this embodiment can display the data access process in different forms from different dimensions based on the results of the full-link analysis of data access. From the data perspective, all nodes that the data passes through from the middle platform to the business system are displayed in the form of topology; from the perspective of microservices, service access interfaces, data, data sources and other information are displayed in the form of charts; from a global perspective, the link details of each access event are displayed in the form of tables and topologies.

[0070] The present invention mainly provides an interface for users in the form of an API, which facilitates users to use the full-link tracking capabilities of the present invention and integrate the results into their own business systems. The general implementation method is to copy the SDK jar file corresponding to the present invention to the environment of the user's business system, and add a dependency on the jar file in the JAVA build path. Users call the required interface according to the provided API description document.

[0071] The system of this embodiment can realize the following functions: query data flow links, query data call status, query application topology, query data ledger, etc.

[0072] In this embodiment, full-link tracking and analysis is performed with the help of the big data platform spark streaming processing components, Kafka, and impala sql analysis components. The main process is as follows:

[0073] S101: Spark consumes the Kafka message queue, periodically obtains the latest offsets in each partition of each topic in Kafka, and then processes each batch according to the set maxRatePerPartition.

[0074] S102: Spark uses the impala component to obtain intermediate connection data, and uses the Spark streaming component and the imapala script to perform association and SQL connection analysis. The main analysis is in three scenarios: First, the complete link of data call is analyzed with the help of the stack SQL information obtained by application monitoring, such as Figure 2 As shown; the second is to use the RDS audit log combined with the monitoring information to analyze the data call complete link, such as Figure 3 As shown; the third is to use the sls audit log to analyze the complete link of the data service call, such as Figure 4 shown.

[0075] Specifically, combined Figure 2 The complete process of analyzing the data call chain with the help of the stack SQL information obtained by application monitoring is as follows:

[0076] Use the stack information monitored by the arms application to obtain the sql information and link identifier traceId, then analyze the table and field information of the database by parsing the sql, and query the microservice IP and microservice unique identifier pid through traceId. Then query the associated database IP and port through pid, and match the rds instance (i.e., cloud database instance) to obtain the project space where the data source is located. Finally, by connecting to the data integration information and dataworks data source information, obtain the detailed information of the entire access link, including the original table name, original library name, target table, target library, execution time, microservice name, IP, business system, business department and other information.

[0077] Combination Figure 3 The specific process of tracking the full-link scenario with the help of RDS audit logs is as follows: analyze the microservice IP, SQL and RDS instance information that accesses the data through RDS audit logs, analyze the table and field information of the database by parsing SQL, obtain detailed information of the instance through RDS instance ID, including instance name and instance IP, and determine the target library and task space ID by comparing the instance IP with the IP of the data source. The microservice IP obtains the microservice name by connecting to the arms application monitoring information, and obtains the business system and department to which the microservice belongs by connecting to the cloud server instance information through the microservice IP. Compare the data integration information through the task space ID and the target library and target table to obtain the source library and source table, and finally obtain the full-link information of the application scenario in which the data integration is synchronized to RDS and the service is directly connected to the RDS database through JDBC.

[0078] Including original table name, original database name, target table, target database, execution time, microservice name, IP, business system, business department and other information.

[0079] Combination Figure 4, the process of tracking the full-link scenario with the help of sls audit logs is as follows: obtain the microservice IP, sql information and cloud database unique identifier of the access data service through sls audit logs, analyze the table and field information of the target database by parsing sql, query the cloud database name and domain name through the cloud database unique identifier, and then obtain the cloud database IP by parsing the domain name, connect to the external data source information, and obtain the target library and task space unique identifier. Compare the data integration information through the target library, target table and task space ID to obtain the original library and source table. The microservice IP obtains the microservice name, the business system where the microservice is located, the business department where the microservice is located, and other information by comparing the arms application monitoring information and the cloud server instance information. Finally, the full-link information of the application scenario that is synchronized to rds through data integration and published as a data service for calling is analyzed, including business system data, resource set ID, resource set name, microservice name, original library name, original table name, target table, target library, whether it is a sensitive table, middle platform type, API name, API domain name, API path, request occurrence time, visitor IP and other information.

[0080] S103: The analyzed data is stored back in Kafka, and two message topics are generated, namely storeInfo and showInfo, one for storage and the other for display.

[0081] S104: The spark streaming component consumes the storage information topic, analyzes the data flow link, and stores the analyzed data in the HDFS or KUDU database;

[0082] S105: The JAVA microservice periodically polls the consumption display information topic to store the data link display information, and pushes it to the front-end micro application through websocket for real-time display.

[0083] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A data call link restoration analysis system based on big data, characterized in that: include: The data collection engine includes a scheduled task collection unit and a flume real-time collection unit. The scheduled task collection unit collects cloud platform resource sets, data sources, and application monitoring interface data through Java scheduled tasks. The collected data is directly stored in the big data platform for subsequent data flow link restoration analysis; the flume real-time collection unit is used to convert log data into json files and then collect and store them in kafka in real time through the flume component; When the data analysis engine performs full-link tracking process analysis, it retrieves data from the big data platform. The data analysis engine is used to receive data in Kafka and convert it into a set data structure. Then, the RDS audit log, SLS audit log, and Arms application stack information are used as analysis entries to perform full-link tracking process analysis, including: The spark streaming component consumes the Kafka message queue, periodically obtains the latest offsets in each partition of each topic in Kafka, and then processes each batch according to the set maxRatePerPartition; The spark streaming component uses the impala component to obtain intermediate connection data, and uses the Spark streaming component and the imapala script to perform association and SQL connection analysis; The analyzed data is stored back in Kafka and two message topics are generated, one for storage and the other for display. The spark streaming component consumes the storage information topic, analyzes the data flow link, and stores the analyzed data in HDFS or KUDU database; The JAVA microservice periodically polls the consumption display information topic to store data link display information, and pushes it to the front-end micro application for real-time display through websocket; The data storage engine is used to store the collected source data, the cleansed and converted data, and the data processed by the data analysis engine in layers; The data display component is used to obtain data processed by the data analysis engine and display the full-link analysis results from three dimensions: data, microservices, and application systems.

2. A data call link restoration analysis system based on big data as claimed in claim 1, characterized in that: The data analysis engine comprises: Impala SQL analysis component, used to perform SQL analysis using RDS audit logs, SLS audit logs, and Arms application stack information as analysis entry points; Spark streaming is a streaming processing component that receives data from Kafka and converts it into the data structure Dstream in Spark streaming. The job submitted by Spark streaming will perform multi-dimensional correlation analysis on the data.

3. A data call link restoration analysis system based on big data as claimed in claim 1, characterized in that: The full-link tracking process analysis includes the following three scenarios: Analyze the complete link of data calls with the help of stack SQL information obtained through application monitoring; Use RDS audit logs combined with monitoring information to analyze data and call the complete link; Use sls audit logs to analyze the complete link of data service calls.

4. A data call link restoration analysis system based on big data as claimed in claim 3, characterized in that: The complete link of analyzing data calls using stack SQL information acquired through application monitoring specifically includes: Use the stack information of application monitoring to obtain SQL information and link identifiers, then analyze the table and field information of the call database by parsing SQL, and query the microservice IP and microservice unique identifier through the link bid; The associated database IP and port are queried through the microservice unique identifier, and the cloud database instance information is matched to obtain the project space where the data source is located; By connecting to data integration information and DataWorks data source information, you can obtain detailed information about the entire access link.

5. A data call link restoration analysis system based on big data as claimed in claim 3, characterized in that: The complete link of calling data by combining RDS audit log with monitoring information analysis data includes: Analyze the microservice IP, SQL, and RDS instance information that access the data through RDS audit logs, analyze the table and field information of the database by parsing SQL, obtain the instance IP through the cloud database unique identifier, and determine the target library and task space ID by comparing the instance IP with the IP of the data source; The microservice IP obtains the microservice name by connecting to the application monitoring information, and obtains the business system and department to which the microservice belongs by connecting to the cloud server instance information through the microservice IP; By querying the data integration table through the unique identifier of the task space and comparing the target library and target table, the data middle platform project space and data middle platform table are obtained, and finally the full-link information of the application scenario in which the data integration is synchronized to the cloud database and the service is directly connected to the cloud database through JDBC is obtained.

6. A data call link restoration analysis system based on big data as claimed in claim 3, characterized in that: The complete link of analyzing data service calls with the help of sls audit logs is as follows: Obtain the microservice IP, SQL information, and cloud database unique identifier of the microservice accessing the data service through the SLS audit log. Analyze the table and field information of the target database by parsing SQL. Query the cloud database name and domain name through the cloud database unique identifier. Then obtain the cloud database IP by parsing the domain name. Connect to the external data source information and obtain the target database and task space unique identifier. Compare and query the data integration table through the external database, external table and task space unique identifier to obtain the middle platform project space and middle table; The microservice IP obtains the microservice name, the business system where the microservice is located, and the business department information by comparing the application monitoring information and the cloud server instance information; Finally, the full-link information is analyzed, synchronized to the cloud database through data integration and published as a data service for calling application scenarios.

7. A data call link restoration analysis system based on big data as claimed in claim 1, characterized in that: The data storage engine is divided into three layers: source layer, sharing layer and analysis layer. The source layer is used to store the collected source data, the sharing layer is used to regularly clean and convert the data in the source layer, and store the results in the sharing layer; the analysis layer is used to store the data processed by the analysis engine.

8. A data call link restoration analysis system based on big data as claimed in claim 7, characterized in that: The specific data for regularly cleaning the source layer are: Filter out the data that does not meet the requirements and hand over the filtering results to the business department to confirm whether it should be filtered out or revised by the business unit before extraction.

9. A data call link restoration analysis system based on big data as claimed in claim 7, characterized in that: The specific data for regularly cleaning the source layer are: Unify the same type of data from different business systems, aggregate the source layer data according to the subject domain granularity of the shared layer, and calculate different data indicators in different business rules.

Images