Data analysis system and computer readable storage medium

Abstract

The embodiment of the invention relates to a data analysis system based on a data center. The data analysis system comprises a display layer, an access layer, a gateway layer, a platform service layer and a data layer, wherein the display layer is configured to receive a service request from user equipment and send the service request to the access layer; the access layer is configured to distribute a service request to the gateway layer based on a preset load balancing setting; the gateway layer is configured to analyze the service request and route the analyzed service request to the data analysis micro-service in the platform service layer; the platform service layer is configured to respond to the analyzed service request, analyze micro-service based on data in the platform service layer, execute service logic corresponding to the service request and generate a service logic result; and the data layer is configured to store data generated and required when the data analysis micro-service executes business logic, and support a data acquisition task and a data management task of the data analysis micro-service.

Description

Technical Field

[0001] The embodiments of the present disclosure generally relate to the technical field of platform system architecture, and more particularly to a data analysis system and a computer-readable storage medium. Background Art

[0002] With the rapid development of the Internet and big data technology, enterprises have an increasing demand for data. As a global professional service organization, we not only need to conduct timely and accurate data analysis, but also rely on data to help employees improve their individual and team efficiency, help project managers efficiently manage project progress, and allow management to gain timely insight into business operation trends. The data center is a research and development project launched to improve the efficiency of data analysis operations.

[0003] Currently, data analysis operation platforms already exist, but there are some problems. First, the efficiency of querying data is not high, and employees need to spend a long time to obtain the required data information. Secondly, when managing the project progress, project managers face the problem of insufficient, timely, and accurate information, which will lead to inaccurate project progress management and affect the progress of work. Furthermore, management needs to view real-time business operation data in order to adjust the operation strategy in a timely manner, but the existing data analysis platform cannot meet its needs.

[0004] In summary, developing a more efficient, convenient and comprehensive data analysis system has become an urgent need for the development of the industry. Summary of the invention

[0005] In response to the above problems, the present disclosure provides a data analysis system based on the data middle platform, which integrates multiple functions such as individual and team usage, project process management, and business operation trend insights. Through this system, employees can more conveniently query data related to individual and team usage efficiency, project managers can manage project processes more efficiently, and management can timely understand business operation trends in order to make more accurate decisions for the development of the enterprise. Ultimately, the successful implementation of this project will help improve the efficiency of data analysis operations and provide strong support and guarantee for the development and progress of the enterprise.

[0006] According to a first aspect of the present disclosure, a data analysis system based on a data middle platform is provided, the system comprising a presentation layer, an access layer, a gateway layer, a platform service layer and a data layer, wherein the presentation layer is applied to a user device and the access layer, the gateway layer, the platform service layer and the data layer are applied to a data background that is communicatively connected to the user device, wherein: the presentation layer is configured to receive a service request from the user device and send the service request to the access layer; the access layer is configured to distribute the service request to the gateway layer based on a preset load balancing setting; the gateway layer is configured to parse the service request and route the parsed service request to a data analysis microservice in the platform service layer; the platform service layer is configured to respond to the parsed service request and, based on the data analysis microservice in the platform service layer, execute the service logic corresponding to the service request and generate the service logic result; and the data layer is configured to store the data generated and required when the data analysis microservice executes the service logic, as well as the data collection tasks and data management tasks that support the data analysis microservice.

[0007] According to a second aspect of the present disclosure, a computing device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method of the first aspect of the present disclosure.

[0008] In a third aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to execute the method of the first aspect of the present disclosure.

[0009] In one embodiment, the platform service layer is further configured to feed back the business logic result to the gateway layer; the gateway layer is further configured to forward the business logic result to the access layer in response to the received business logic result; the access layer is further configured to distribute the business logic result to the presentation layer in sequence based on a preset load balancing rule; and the presentation layer is further configured to present the business logic result on the user device in response to the business logic result.

[0010] In one embodiment, the data analysis microservice is configured to call the API interface in response to the business request to execute the corresponding business logic and generate business logic results; and feed back the business logic results to the gateway layer, wherein the data analysis microservice includes: home page project microservice, participation project microservice, administrative project microservice, utilization project microservice, indicator project microservice, hub project microservice and account project microservice.

[0011] In one embodiment, the homepage project microservice is configured to connect gateways of different user devices and to parse and distribute messages from gateways of different user devices, thereby realizing specific message transmission between different gateways; the participating project microservice is configured to collect and manage project data reports based on the data layer; the administrative project microservice is configured to collect and manage enterprise financial data, management data, operating data, report data, and department project time utilization data based on the data layer; the utilization project microservice is configured to collect and manage personal project time utilization data and consulting project time utilization data based on the data layer and the participating project microservice; the indicator project microservice is configured to collect and manage personal performance data based on the data layer, the participating project microservice, and the utilization project microservice; the hub project microservice is configured to display the data collected and managed by the data layer, the participating project microservice, the utilization project microservice, and the indicator project microservice using a custom table format; and the account project microservice is configured to analyze the data collected and managed by the participating project microservice, the utilization project microservice, and the indicator project microservice using the account data of the data layer.

[0012] In one embodiment, the data layer includes: a real-time data warehouse, a relational database, a distributed cache database, and a distributed file storage repository, so as to store and process the business requests and the data to be processed in the data background.

[0013] In one embodiment, the real-time data warehouse is configured to: determine whether the data to be processed can be collected in real time; in response to the data to be processed being able to be collected in real time, perform real-time stream processing on the data to be processed; and in response to the data to be processed not being able to be collected in real time, perform offline batch processing on the data to be processed; the distributed cache database is configured to: determine whether the data to be processed is hotspot access data; and in response to the data to be processed being hotspot access, use a three-sentinel cluster mode of one master and two slaves to store the data to be processed; the relational database is configured to store and process structured data in the data to be processed; and the distributed file storage repository is configured to store and process unstructured data in the data to be processed.

[0014] In one embodiment, the data layer also includes a data backup microservice and a recovery mechanism microservice, wherein the data backup microservice is configured to: regularly back up data to prevent data loss or damage, wherein the real-time data warehouse uses a shared file system mount backup, the distributed file repository uses a scheduled script backup, the distributed cache database uses memory snapshots and log AOF hot and cold backups, and the relational database uses SCP remote backup; the data recovery microservice is configured to quickly restore data from the backup when the data is lost or damaged, so as to ensure the continuous operation of the data analysis system and the integrity of the data.

[0015] In one embodiment, the data to be processed includes project data, time data, business data, and target object data.

[0016] In one embodiment, the data display layer is composed of a loosely coupled multi-module front-end Javascript framework, and the data display layer is built based on the React framework.

[0017] In one embodiment, the system also includes: an infrastructure layer, configured to adopt container service ACK technology to provide a network virtual server for deploying and running the presentation layer, access layer, gateway layer, platform service layer and data layer in the data analysis system.

[0018] By integrating multiple microservices, data can be quickly processed and analyzed, which improves work efficiency and decision-making quality. At the same time, the system adopts a front-end and back-end separation architecture and a microservice framework, which is highly flexible and scalable and can adapt to the needs of different business scenarios. In addition, the system also supports multiple data interfaces and protocols, which facilitates docking and integration with different data sources, further improving the practicality and versatility of the system.

[0019] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements.

[0021] Figure 1 A schematic diagram of a data analysis system according to an embodiment of the present invention is shown.

[0022] Figure 2 A schematic diagram of another data analysis system according to an embodiment of the present invention is shown.

[0023] Figure 3 A block diagram of a data analysis system according to an embodiment of the present invention is shown.

[0024] Figure 4 A block diagram of an electronic device 400 according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0025] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0026] As used herein, the term "including" and its variations mean open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "based at least in part on". The terms "an example embodiment" and "an embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0027] Figure 1 FIG. 2 shows a schematic diagram of a data analysis system according to an embodiment of the present invention. Figure 1 As shown, the data analysis system 100 based on the data middle platform includes a presentation layer 102, an access layer 104, a gateway layer 106, a platform service layer 108, and a data layer 110. The presentation layer in the data analysis system 100 is applied to the user device, and the access layer, the gateway layer, the platform service layer, and the data layer are applied to the data background that is connected to the user device for communication.

[0028] Specifically, the user device described in the embodiment of the present invention can be a mobile phone, a desktop computer, a laptop computer or a cloud server and other devices. The display layer can be a web front end or an APP front end. The display layer is composed of a loosely coupled multi-module front-end javascript framework, and the display layer can be built based on the VUE framework or React. The framework mode is adopted and the JQuery, Async, and RequireJS class libraries are used to develop the front-end display layer. The core control adopts the Wisdge basic control library (including tree, list, table, scroll, splitter, dropMenu, buttonGroup, etc.). All interactions use asynchronous mode to avoid the use of synchronous locks. The multi-module front-end javascript framework includes multiple highly cohesive service plug-ins and service modules. The service plug-ins include communication module plug-ins, message module plug-ins, multimedia service plug-ins, knowledge base plug-ins and log plug-ins. The service module includes a dynamic page rendering module, an instance management module and a data service module.

[0029] Specifically, the data backend described in the embodiments of the present invention refers to the resources at the bottom of the backend, which may include infrastructure resource pools and industry resource pools, specifically physical resources, cloud resources, and industry resources. Among them, physical resources include servers, switches, storage, etc.; cloud resources include cloud servers, cloud storage, etc.; industry resources may include various network elements, edge gateways, etc.

[0030] In this embodiment, the presentation layer is configured to receive a service request from a user device and send the service request to the access layer. The access layer is configured to distribute the service request to the gateway layer based on a preset load balancing rule. Load balancing is to ensure that in a high-concurrency scenario, each microservice can process user requests evenly and improve the overall performance and reliability of the system.

[0031] In one embodiment, load balancing may include an access layer determining a microservice architecture of a platform service layer, where each microservice may deploy multiple instances to improve processing power and fault tolerance. The access layer selects appropriate load balancing strategies, such as polling algorithms, weighted polling algorithms, minimum number of connections algorithms, minimum response time algorithms, etc., based on business requirements and system characteristics. The access layer uses a service registry (such as Eureka, Consul, Nacos, etc.) to register and discover microservices. Microservice instances register with the registry at startup, and the registry maintains a list of service instances. When a request is made to access a microservice, the list of available service instances is queried through the registry.

[0032] The load balancer at the access layer (such as Nginx, HAProxy, etc.) is responsible for distributing user requests to microservice instances. Configure the routing rules of the load balancer according to the selected load balancing strategy. For example, when using the polling strategy, the load balancer will distribute requests to the microservice instances in turn; when using the minimum number of connections strategy, it will prioritize the distribution of requests to the instance with the least number of current connections. The load balancer will perform health checks on the microservice instances regularly or in real time to ensure that only instances that are working normally participate in request processing. Health checks usually include heartbeat detection, interface call detection, etc. Instances that fail the detection will be temporarily removed from the service list.

[0033] The access layer dynamically expands or shrinks the number of microservice instances based on the system load. For example, during peak hours, the number of microservice instances can be automatically increased to cope with high concurrent requests; during off-peak hours, the number of instances can be reduced to save resources. The load of microservice instances is monitored in real time, including indicators such as CPU usage, memory usage, and request response time. The access layer can set alarm thresholds. When the load of an instance exceeds the threshold, an alarm notification is triggered so that timely measures can be taken to intervene. When a microservice instance fails, the load balancer will forward the request to other normal instances to ensure the availability and reliability of the system. At the same time, for the failed instance, automatic recovery or restart can be performed to restore it to normal as soon as possible.

[0034] Through the above steps and measures, the platform service layer can achieve efficient load balancing and ensure the stability and performance of the system in high-concurrency scenarios.

[0035] In one embodiment, the gateway layer is configured to parse the business request and route the parsed business request to the microservice in the platform service layer. The gateway layer forwards the business request to the platform service layer in response to the business request. The platform service layer includes predefined microservices. The platform service layer responds to the business request, executes the business logic corresponding to the business request through the microservice, generates the business processing result of the business request, and sends the business processing result back to the gateway layer. The gateway layer responds to the business processing result and sends the business processing result back to the terminal device. The presentation layer responds to the business processing result and displays the business processing result on the user device.

[0036] As described above, the platform service layer includes the insurance business microservice architecture and the data analysis microservice architecture. The platform service layer responds to the business request, executes the business logic corresponding to the business request through the insurance business microservice architecture and the data analysis microservice architecture, generates the business processing result of the business request, and sends the business processing result back to the access gateway layer. The access gateway layer responds to the business processing result and sends the business processing result back to the terminal device. The front-end display layer responds to the business processing result and displays the business processing result on the terminal device.

[0037] In one embodiment, the platform service layer adopts a microservice architecture to split the system into multiple independent, independently deployable and scalable microservices. Each microservice is responsible for a specific business function and communicates and collaborates with other microservices through an API interface. This architecture makes the system more flexible, scalable and maintainable. The platform service layer integrates a variety of microservices based on the Spring Cloud framework, including homepage project microservices, participation project microservices, administrative project microservices, utilization project microservices, indicator project microservices, hub project microservices and account project microservices. These microservices call the API interface in response to business requests to execute the corresponding business logic, generate business logic results and feed back to the gateway layer. The homepage project microservice is used to connect the gateways of different user devices and parse and distribute messages. The participation project microservice is used to collect and manage project data reports. The administrative project microservice is used to collect and manage enterprise financial data, management data, operating data, report data, and department project time utilization data. The utilization project microservice is used to collect and manage personal project time utilization data and consulting project time utilization data. The indicator project microservice is used to collect and manage personal performance data. The hub project microservice is used to display data in a custom table format. The account project microservice is used to analyze data using account data.

[0038] In one embodiment, the platform service layer may also include a data processing microservice, which may include pre-processing tasks such as data cleaning, conversion, and aggregation to provide high-quality data sources for data analysis. The platform service layer may also include a report generation microservice, which obtains data from the data layer and generates corresponding reports, such as statistical reports, trend analysis reports, etc., based on user requests. The platform service layer may also include an algorithm operation microservice, which is responsible for running various data analysis algorithms, such as machine learning algorithms, data mining algorithms, etc., to provide users with intelligent data analysis results. The platform service layer may also include a permission management microservice, which manages user permissions and roles to ensure the security of the system.

[0039] The platform service layer interacts with the data layer, and obtains data from the data layer or stores the processing results in the data layer through data interaction components. These interaction components include database connection pools, data access objects (DAOs), etc., which are responsible for encapsulating the underlying data access logic and providing a unified data access interface for the upper-layer business logic. The platform service layer uses service registration centers (such as Eureka, Consul, etc.) to manage the registration and discovery of microservices to ensure dynamic discovery and calling between services. The platform service layer automatically distributes requests to different service instances based on the load of microservices to improve the throughput and response speed of the system. You can also use monitoring tools (such as Prometheus, Grafana, etc.) to monitor microservices in real time, set alarm thresholds, and issue alarm notifications in time when services are abnormal.

[0040] The platform service layer can also authenticate and verify permissions for requests to access the platform service layer, ensuring that only authorized users can access the corresponding services. Encrypted storage and transmission of sensitive data, desensitization of query results, and protection of user data security. User access and operation logs are recorded to facilitate auditing and tracing of issues.

[0041] Through the design and implementation of the above structure, the platform service layer can provide stable, efficient and scalable business logic processing and data service support for the data analysis system, ensuring that the system can meet various complex data analysis needs.

[0042] In one embodiment, the data layer is configured to store the data generated and required when the data analysis microservice executes the business logic, and to support the data collection tasks and data management tasks of the data analysis microservice. The data layer is responsible for storing the data generated and required when the data analysis microservice executes the business logic, and supporting the data collection tasks and data management tasks. The data layer includes a real-time data warehouse microservice, a relational database microservice, a distributed cache database microservice, and a distributed file repository microservice. The real-time data warehouse microservice performs real-time stream processing or offline batch processing according to whether the data to be processed can be collected in real time. The distributed cache database microservice uses a three-sentinel cluster mode of one master and two slaves for hot access data. The relational database microservice stores and processes structured data. The distributed file repository microservice stores and processes unstructured data. In addition, the data layer also includes a data backup microservice and a recovery mechanism microservice to ensure the security and integrity of the data. The real-time data warehouse uses a shared file system mount backup, the distributed file repository uses a scheduled script backup, the distributed cache database uses snapshot RDB and log AOF hot and cold backup, and the relational database uses SCP remote backup

[0043] Figure 2 FIG. 2 shows a schematic diagram of another data analysis system according to an embodiment of the present invention. Figure 2 As shown, the data analysis system also includes an infrastructure layer 212, which is configured to adopt container service ACK technology to provide a network virtual server for deploying and running the presentation layer, access layer, gateway layer, platform service layer and data layer in the system and providing underlying technical support for the data layer and platform service layer.

[0044] The infrastructure layer structure of Alibaba Cloud Container Service for Kubernetes (ACK) is the foundation of the underlying resources that the container service relies on, providing stable and reliable computing, network and storage capabilities for the entire system. The nodes in the ACK cluster of the infrastructure layer 212 are the servers that actually run containerized applications. Nodes can be virtual machine instances or physical servers with Docker Engine installed for deploying and managing containers. A node pool is a set of nodes with the same configuration in a cluster. ACK supports two types: node pools and managed node pools, and can flexibly adjust the number and configuration of nodes according to business needs. The network resources of the infrastructure layer 212 include a private network VPC (Virtual Private Cloud): the ACK cluster is deployed in the VPC to provide a private and isolated network environment. Users can fully control their own VPC, including configurations such as IP address ranges, routing tables and gateways. The infrastructure layer 212 can implement load balancing (Load Balancer): used to distribute traffic entering the ACK cluster to multiple container instances to improve the scalability and availability of the system. The cloud disk of the infrastructure layer 212 provides persistent storage capabilities for ACK nodes, supports multiple storage types and specifications, and meets the data storage requirements of different applications. Storage volume of the infrastructure layer 212: The storage volume in Kubernetes is an abstraction of persistent storage, allowing containers to access storage systems outside the cluster, such as cloud disks, network file systems, etc. The security group of the infrastructure layer 212 is used to divide security domains in the cloud, with state detection and packet filtering capabilities to protect the ACK cluster from unauthorized access and attacks. The infrastructure monitoring of the infrastructure layer 212 provides monitoring capabilities for the infrastructure layer, including performance monitoring of host nodes and network infrastructure components, to ensure the stable operation of the infrastructure. Cluster Autoscaler (CA) of the infrastructure layer 212: As a resource layer elastic component, it automatically adjusts the number of nodes according to the resource usage of the cluster to meet the application's demand for computing resources. The Agent program of the infrastructure layer 212 is installed on each node and is responsible for the communication and management between the node and the ACK cluster.

[0045] The infrastructure layer structure of Alibaba Cloud Container Service ACK provides powerful computing, networking, storage, and security capabilities for containerized applications, supports elastic scaling and automated management, and ensures stable and efficient operation of applications. Through reasonable configuration and management, ACK can fully leverage its advantages in cloud-native application deployment and management.

[0046] Figure 3FIG. 4 shows a block diagram of a data analysis system according to an embodiment of the present invention. Figure 3 As shown, the system includes a presentation layer, an access layer, a gateway layer, a platform service layer, a public technology layer, a data layer, and an infrastructure layer. The public technology layer is a public technology call layer that can be used for other layers, including distributed caching, distributed registration, distributed scheduling, and distributed configuration.

[0047] As the entry point for user access, the data analysis system provides unified data query, report display and custom analysis functions. The data portal supports multiple query methods, including graphical interface query, SQL query, etc., to meet the data application needs of different users. The data analysis system is responsible for data quality monitoring, data standard formulation, data security and sharing and other governance work. By setting data quality standards and establishing data security mechanisms, the accuracy and security of data are ensured.

[0048] The data analysis system provides data storage, processing and analysis capabilities. The platform is based on a distributed architecture and uses big data processing technology to support real-time processing and analysis of massive data.

[0049] The front end of the data analysis system uses a modern front-end framework, such as React or Vue.js, to provide a rich user interface and interactive experience. The back end uses Java as the development language and is built on the Spring Cloud microservice framework to achieve decoupling and independent deployment of each service module.

[0050] The data analysis system uses the OpenFegin framework to call microservices to achieve communication and collaboration between services. At the same time, Nacos is used as a distributed registration center and configuration center to achieve automatic registration, discovery, and configuration management of services.

[0051] The data analysis system uses Alibaba's self-developed one-stop real-time data warehouse engine Hologres as the data storage and query service. Hologres supports real-time data writing and efficient query, meeting the requirements of data analysis for real-time and accuracy.

[0052] Hot data cache of the data analysis system: In order to speed up product response time, cloud Redis is used to store hot access data. When users request hot data, they can directly obtain it from the Redis cache, reducing the access pressure on the backend database and improving the system's response speed.

[0053] At the same time, the development of the data analysis system of the present invention introduces the DevOps concept, and improves the efficiency of system development, testing and online through automated deployment, monitoring and operation and maintenance. At the same time, a continuous integration and continuous delivery process is established to ensure the quality of the code and the stability of the system.

[0054] The data analysis system works closely with the business department to clarify the specific needs of data analysis and operations, including data sources, data formats, query methods, etc. Based on the results of the demand analysis, the technical architecture and functional modules of the system are designed and coded for implementation. During the development process, attention is paid to the readability, maintainability and scalability of the code. The developed system is fully tested and verified, including functional testing, performance testing, security testing, etc. Ensure that the system meets business requirements and has good stability and security. Deploy the system to the production environment through an automated deployment process, and conduct continuous monitoring and operation and maintenance. At the same time, the system is continuously optimized and improved based on user feedback and business changes.

[0055] Through the introduction of the above specific implementation methods, it can be seen that the present invention constructs an efficient, convenient and comprehensive data analysis and operation platform by adopting microservice architecture, front-end and back-end separation, distributed storage and other technical means, which effectively solves the problems existing in the original system and improves the efficiency of data analysis operations.

[0056] Figure 4 4 shows a schematic block diagram of an example electronic device 400 that can be used to implement an embodiment of the present disclosure. Figure 2 The computing device 40 shown can be implemented by an electronic device 400. As shown, the electronic device 400 includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 402 or computer program instructions loaded from a storage unit 408 to a random access memory (RAM) 403. In the random access memory 403, various programs and data required for the operation of the electronic device 400 can also be stored. The central processing unit 401, the read-only memory 402, and the random access memory 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0057] Multiple components in the electronic device 400 are connected to the input / output interface 405, including: an input unit 406, such as a keyboard, a mouse, a microphone, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a disk, an optical disk, etc.; and a communication unit 409, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 409 allows the device 400 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0058] The various systems and microservices described above may be executed by the central processing unit 401. For example, in some embodiments, the systems and microservices may be implemented as computer software programs, which are tangibly contained in a machine-readable medium, such as a storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 400 via the read-only memory 402 and / or the communication unit 409. When the computer program is loaded into the random access memory 403 and executed by the central processing unit 401, one or more actions in the systems and microservices described above may be performed.

[0059] The present disclosure relates to methods, apparatuses, systems, electronic devices, computer-readable storage media and / or computer program products. The computer program products may include computer-readable program instructions for executing various aspects of the present disclosure.

[0060] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0061] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge computing devices. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing / processing device.

[0062] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0063] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0064] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0065] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0066] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0067] Those skilled in the art should understand that the present invention is not limited to the above-mentioned embodiments, and the present invention can be implemented in many other forms without departing from its purpose and scope. Therefore, the examples and embodiments shown are regarded as illustrative rather than restrictive, and the present invention may cover various modifications and substitutions without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A data analysis system based on a data middle platform, characterized in that: The system comprises a presentation layer, an access layer, a gateway layer, a platform service layer and a data layer, wherein the presentation layer is applied to a user device and the access layer, the gateway layer, the platform service layer and the data layer are applied to a data background in communication connection with the user device, wherein: The presentation layer is configured to receive a service request from the user equipment and send the service request to the access layer; The access layer is configured to distribute service requests to the gateway layer based on a preset load balancing setting; The gateway layer is configured to parse the business request and route the parsed business request to the data analysis microservice in the platform service layer; The platform service layer is configured to respond to the parsed business request and, based on the data analysis microservice in the platform service layer, execute the business logic corresponding to the business request to generate the business logic result; and The data layer is configured to store the data generated and required when the data analysis microservice executes business logic, and to support the data collection tasks and data management tasks of the data analysis microservice.

2. The data analysis system according to claim 1, characterized in that: The platform service layer is further configured to feed back the business logic result to the gateway layer; The gateway layer is further configured to forward the business logic result to the access layer in response to the received business logic result; The access layer is further configured to distribute the business logic results to the presentation layer in sequence based on a preset load balancing rule; as well as The presentation layer is further configured to present the business logic result on the user device in response to the business logic result.

3. The data analysis system according to claim 2, characterized in that: The data analysis microservice is configured to call the API interface in response to the business request to execute the corresponding business logic and generate a business logic result; And the business logic results are fed back to the gateway layer, wherein the data analysis microservices include: home page project microservices, participation project microservices, administrative project microservices, utilization project microservices, indicator project microservices, hub project microservices and multiple items of account project microservices.

4. The data analysis system according to claim 3, characterized in that: The homepage project microservice is configured to connect gateways of different user devices and parse and distribute messages from gateways of different user devices, thereby realizing specific message transmission between different gateways; The project participation microservice is configured to collect and manage project data reports based on the data layer; The administrative project microservice is configured to collect and manage enterprise financial data, management data, operating data, report data, and department project work time utilization data based on the data layer; The utilization project microservice is configured to collect and manage personal project work time utilization data and consulting project work time utilization data based on the data layer and the participation project microservice; The indicator project microservice is configured to collect and manage personal performance data based on the data layer, the participation project microservice, and the utilization project microservice; The hub project microservice is configured to display the data collected and managed by the data layer, the participating project microservice, the utilizing project microservice, and the indicator project microservice using a custom table format; as well as The account project microservice is configured to use the account data of the data layer to analyze the data collected and managed by the participation project microservice, the utilization project microservice, and the indicator project microservice.

5. The data analysis system according to claim 4, characterized in that: The data layer includes: a real-time data warehouse, a relational database, a distributed cache database and a distributed file storage repository, which are used to store and process the business requests and the data to be processed in the data background.

6. The data analysis system according to claim 5, characterized in that: The real-time data warehouse is configured to: determine whether the data to be processed can be collected in real time; and perform real-time stream processing on the data to be processed in response to the data to be processed being collected in real time; and performing offline batch processing on the data to be processed in response to the data to be processed being unable to be collected in real time; The distributed cache database is configured to: determine whether the data to be processed is hotspot access data; and in response to the data to be processed being hotspot access, adopt a three-sentinel cluster mode of one master and two slaves to store the data to be processed; The relational database is configured to store and process structured data in the data to be processed; as well as The distributed file storage repository is configured to store and process unstructured data in the data to be processed.

7. The data analysis system according to claim 6, characterized in that: The data layer also includes a data backup microservice and a recovery mechanism microservice. The data backup microservice is configured to: regularly back up data to prevent data loss or damage, wherein the real-time data warehouse uses a shared file system mount backup, the distributed file repository uses a scheduled script backup, the distributed cache database uses memory snapshots and log AOF hot and cold backups, and the relational database uses SCP remote backup; The data recovery microservice is configured to quickly restore data from backup when data is lost or damaged, so as to ensure the continuous operation of the data analysis system and the integrity of the data.

8. The data analysis system according to claim 7, characterized in that: The data to be processed includes project data, time data, business data and target object data.

9. The data analysis system according to claim 8, characterized in that: The data display layer is composed of a loosely coupled multi-module front-end Javascript framework, and the data display layer is built based on the React framework.

10. The data analysis system according to any one of claims 1 to 7, characterized in that: The system further comprises: The infrastructure layer is configured to adopt the container service ACK technology to provide a network virtual server for deploying and running the presentation layer, access layer, gateway layer, platform service layer and data layer in the data analysis system.

Images