Service management and control system based on big data

By designing a service management and control system based on big data, the problem of inefficient data processing and maintenance in the vertical e-commerce industry is solved, efficient data management and secure sharing are realized, API calls are ensured and user satisfaction is improved.

CN118940263AInactive Publication Date: 2024-11-12苏州当至创网络科技有限公司
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410784990.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-18
Publication Date
2024-11-12
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing technology cannot effectively meet the processing and maintenance needs of large-scale structured and unstructured data in the vertical e-commerce industry, resulting in inefficient data service management and control systems and unable to meet the data processing and management needs of enterprises.

Method used

Design a service management and control system based on big data, including task scheduling control, gateway optimization, analysis control and data transmission storage, and task scheduling through the Quartz framework, API gateway management, traffic control, funnel analysis and blockchain storage to ensure data security and reliability.

Benefits of technology

It improves the management capabilities of the data service management system, realizes the fast, low-cost openness and sharing of data, ensures the security of API calls, prevents tampering and malicious use, and improves user satisfaction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118940263A_ABST
    Figure CN118940263A_ABST
Patent Text Reader

Abstract

The invention discloses a service management and control system based on big data. An operation method of the system comprises the steps that task scheduling control processing of the data service management and control system is carried out; performing gateway optimization processing on the data service management and control system; analyzing, controlling and processing the data service management and control system; data transmission and storage of the data service management and control system are carried out; the task scheduling control processing of the data service management and control system comprises the following steps: completing task scheduling processing through a Quartz framework, transmitting a scheduling task to the Quartz framework when the data service management and control system is started, and then controlling the framework to schedule and execute a corresponding task according to configured parameter data. The gateway optimization processing of the data service management and control system comprises the following steps: carrying out optimization control management on an API gateway management end, and after a user requests to an API gateway, mapping the API gateway into a request address of a back-end service according to parameter information configured by the gateway. The system has the characteristics of intelligent management and control processing and high efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of big data technology, and in particular to a service management and control system based on big data. Background Art

[0002] In the era of rapid development of the Internet, accompanied by the advancement of big data technology, the continuous development and deepening of social informatization, and the construction of IT Internet informatization has penetrated into all walks of life, more and more information is transmitted and interacted between the real and virtual worlds in the form of data. In the relevant fields of the vertical e-commerce industry, after years of accumulation and development, a large amount of structured and unstructured data has been generated, such as user behavior logs, order data, transaction data, business data, etc. The data assets accumulated by current e-commerce companies have the characteristics of multi-source heterogeneity, huge data volume, and both real-time system data and historical data must be taken into account. However, existing technologies can no longer meet the current needs of enterprise data processing and maintenance. Therefore, it is necessary to design a service control system based on big data with intelligent control processing and high efficiency. Summary of the invention

[0003] The purpose of the present invention is to provide a service management and control system based on big data to solve the problems raised in the above background technology.

[0004] In order to solve the above technical problems, the present invention provides the following technical solutions: a service management and control method based on big data, comprising: Carry out task scheduling and control processing of the data service management and control system; Optimize the gateway of the data service management and control system; Conduct analysis and control of data service management and control systems; Carry out data transmission and storage of data service management and control system; The task scheduling control process of the data service management and control system includes: The task scheduling process is completed through the Quartz framework. When the data service management and control system is started, the scheduling task is transmitted to the Quartz framework, and then the framework is controlled to schedule and execute the corresponding task according to the configured parameter data.

[0005] According to the above technical solution, the gateway optimization processing of the data service management and control system includes: Optimize and control the API gateway management end. After the user requests to the API gateway, it is mapped to the request address of the backend service according to the parameter information configured by the gateway, and then the backend service is called to obtain the request result. The API gateway backend service type also supports HTTP and HTTPS. The backend address is the actual request information of the API interface service on the backend server. After the user generates a key pair and binds it to the API, the "HmacSHA256" encryption algorithm is used to encrypt and sign the parameters.

[0006] According to the above technical solution, the analysis and control processing of the data service management and control system includes: When a user accesses the API for the first time, the SET command is first called to set the initialization parameters according to the user's own needs. Then, for each access to the API, the user's traffic control policy configuration is used to check whether the user's access times have reached the access limit within the corresponding time interval.

[0007] According to the above technical solution, the analysis and control processing of the data service management and control system further includes: When the user calls the corresponding API, if the set alarm rule is triggered, the alarm information will be generated and pushed to the KafKa message queue; After the user configures the funnel parameters through the funnel analysis model, a data analysis query task is generated according to the user-configured parameters, and the task request is submitted to the Worker node for execution to obtain the analysis result information.

[0008] According to the above technical solution, the data transmission and storage of the data service management and control system includes: After receiving the data information that has completed the funnel analysis and processing, it will be transmitted to the corresponding workbench for management personnel to review and analyze; The data information processed in the data service management and control system is stored in the blockchain, which effectively ensures the security of the data information and avoids its loss or tampering.

[0009] According to the above technical solution, a service management and control system based on big data includes: The scheduling management module is used to perform scheduling control management of the data service management and control system; Analysis and processing module, used for operation analysis and processing of data service management and control system; The transmission and storage module is used for transmission and storage of data information.

[0010] According to the above technical solution, the scheduling management module includes: Task scheduling module, used for task scheduling management of data service control system; Gateway management module, used to manage the gateway of the data service management and control system; The processing control module is used to perform operation processing control of the gateway.

[0011] According to the above technical solution, the analysis and processing module includes: Traffic control module, used for traffic control of data service management and control system; Warning processing module, used for warning analysis and processing of data information; The analysis control module is used to control and manage the funnel analysis model.

[0012] According to the above technical solution, the transmission storage module includes: An acquisition and transmission module is used to acquire and transmit data information; The data storage module is used to store data information on the blockchain.

[0013] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: the present invention, by providing a scheduling management module, an analysis and processing module, and a transmission and storage module, can improve the ability of users related to the data service management and control system to manage APIs. Users can quickly and cost-effectively realize data openness and sharing services through simple operations, effectively improving user satisfaction, ensuring the security and reliability of API calls, and preventing user API requests from being tampered with or used maliciously to a great extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings: Figure 1 This is a flow chart of a service management and control method based on big data provided by Embodiment 1 of the present invention; Figure 2 This is a module structure diagram of a service management and control system based on big data provided in Example 2 of the present invention. DETAILED DESCRIPTION

[0015] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0016] Embodiment 1: Figure 1 This is a flowchart of a service control method based on big data provided in Embodiment 1 of the present invention. This embodiment can be applied to a data service control system. The method can be executed by a service control system based on big data provided in Embodiment 1 of the present invention. The system is composed of multiple software and hardware modules, such as Figure 1 As shown, the method specifically comprises the following steps: S101, performing task scheduling control processing of the data service management and control system; Exemplarily, in an embodiment of the present invention, task scheduling is completed through the Quartz framework. When the data service management and control system is started, the scheduled task is transmitted to the Quartz framework, and then the framework is controlled to schedule the execution of the corresponding task according to the configured parameter data. Since various types of analysis, calculation and query tasks are common in data service systems, unified planning and task scheduling management of the scheduled task scheduling function are required. In this step, after the user selects the corresponding task template according to his or her needs, an add input page pops up, and the relevant information of the specific task is entered on the page, mainly including the task name, operation mode, task start time, end time, task configuration parameters, and whether the task is started by default, etc., and the information is verified. If successful, a task information is inserted. If the task starts by default, the insertion is successful, and the task scheduling execution can be completed through the Quartz framework as planned, effectively improving the efficiency and accuracy of task scheduling execution processing.

[0017] S102, performing gateway optimization processing on the data service management and control system; Exemplarily, in an embodiment of the present invention, the API gateway management end is optimized, controlled and managed. After the user makes a request to the API gateway, it is mapped to the request address of the backend service according to the parameter information configured in the gateway, and then the backend service is called to obtain the request result, so that the API gateway backend service type also supports both HTTP and HTTPS types, and the backend address is the actual request information of the API interface service on the backend server; since the API gateway management end focuses on implementing the basic management functions such as definition, release, configuration, and authorization of the entire life cycle of the API data interface, it is a unified API management entrance for gateway users and developers. Therefore, through this step, the ability of users related to the data service management and control system to manage the API can be improved, and users can quickly and cost-effectively implement data openness and sharing services through simple operations, effectively improving user satisfaction.

[0018] After the user generates a key pair and binds it to the API, the "HmacSHA256" encryption algorithm is used to encrypt and sign the parameters. In this step, after the user generates a key pair and binds it to the API, the gateway obtains the HTTP request header information involved in the signature and concatenates the signature string address from the paramsMap, and generates a string to be signed from the parameter information, request method, URL, etc. in the headers through an instance of the StringBuffer class. The gateway obtains the user's Secret key based on the user Key, generates a signature string, and finally compares it with the Signature signature in the Headers. If the comparison is successful, it means that the user's request is a legal call. If the comparison is unsuccessful, it means that the request parameter information has been tampered with or other related problems, and the user is prompted that the API call is unsuccessful, indicating that the user's key pair information may be leaked at this time, and further warns the user to reset the key pair information to ensure the security and reliability of the API call, and to prevent the user's API request from being tampered with or used maliciously to a great extent.

[0019] S103, performing analysis and control processing of the data service management and control system; Exemplarily, in an embodiment of the present invention, when a user accesses an API for the first time, the SET command is first called to set initialization parameters according to the user's own needs. Then, for each access to the API, the user's traffic control policy configuration is used to verify whether the user's access times have reached the access limit within the corresponding time interval. In this step, the parameter configuration must be reinitialized each time a new time interval is entered. For access control of the time interval, the number of accesses per minute, per hour, and per day can be set, wherein the traffic control policy includes control of the number of accesses to each API, each user, and each application. Through this step, the back-end services of the data service management and control system can be protected, and at the same time, the stable operation of the business can be effectively guaranteed. The purpose of configuring according to the different needs of users and guaranteeing user rights can also be achieved.

[0020] When the user calls the corresponding API, if the set alarm rule is triggered, an alarm message will be generated and pushed to the KafKa message queue. In this step, the administrator sets the alarm rule based on the selected API. First, the getApiInfo() method is used to determine the specific API group and the API interface under the group. Then, the specific API rule information is set, which includes the rule name, alarm level, and rule description. The alarm level is divided into three basic warning levels: "general warning", "important warning", and "serious warning". The rule description includes three types of monitoring, namely "API average response time", "API total number of requests", and "number of request errors". Finally, after the setting is completed, the saveMonitorRule() method is called to save the rule information and synchronize it to the Redis cache so that the latest alarm rule can take effect when the alarm rule is enabled.

[0021] After the user configures the funnel parameters through the funnel analysis model, a data analysis query task is generated according to the user-configured parameters, and the task request is submitted to the Worker node for execution to obtain the analysis result information. In this step, the funnel analysis query parameters include the query range of the date and the specified event stream and the conversion cycle of the event stream, etc., and because the funnel model can monitor and analyze the operation process and the conversion rate of each layer, and discover the weak links in the operation, this step is conducive to the analysis and optimization of the subsequent data service management and control system.

[0022] S104, performing data transmission and storage of the data service management and control system; Exemplarily, in an embodiment of the present invention, after receiving the data information that has completed the funnel analysis processing, it is transmitted to the corresponding workbench for the management personnel to review and analyze; The data information processed in the data service management and control system is stored in the blockchain, which effectively ensures the security of the data information and avoids its loss or tampering.

[0023] Embodiment 2: Embodiment 2 of the present invention provides a service management and control system based on big data. Figure 2 A schematic diagram of the module structure of a service management and control system based on big data provided in the second embodiment, such as Figure 2 As shown, the system includes: The scheduling management module is used to perform scheduling control management of the data service management and control system; Analysis and processing module, used for operation analysis and processing of data service management and control system; The transmission and storage module is used for transmission and storage of data information.

[0024] In some embodiments of the present invention, the scheduling management module includes: Task scheduling module, used for task scheduling management of data service control system; Gateway management module, used to manage the gateway of the data service management and control system; The processing control module is used to perform operation processing control of the gateway.

[0025] In some embodiments of the present invention, the analysis and processing module includes: Traffic control module, used for traffic control of data service management and control system; Warning processing module, used for warning analysis and processing of data information; The analysis control module is used to control and manage the funnel analysis model.

[0026] In some embodiments of the present invention, the transmission storage module includes: An acquisition and transmission module is used to acquire and transmit data information; The data storage module is used to store data information on the blockchain.

[0027] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0028] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A service management and control method based on big data, characterized in that: include: Carry out task scheduling and control processing of the data service management and control system; Optimize the gateway of the data service management and control system; Conduct analysis and control of data service management and control systems; Carry out data transmission and storage of data service management and control system; The task scheduling control process of the data service management and control system includes: The task scheduling process is completed through the Quartz framework. When the data service management and control system is started, the scheduling task is transmitted to the Quartz framework, and then the framework is controlled to schedule and execute the corresponding task according to the configured parameter data.

2. A service management and control method based on big data according to claim 1, characterized in that: The gateway optimization processing of the data service management and control system includes: Optimize and control the API gateway management end. After the user requests to the API gateway, it is mapped to the request address of the backend service according to the parameter information configured by the gateway, and then the backend service is called to obtain the request result. The API gateway backend service type also supports HTTP and HTTPS. The backend address is the actual request information of the API interface service on the backend server. After the user generates a key pair and binds it to the API, the "HmacSHA256" encryption algorithm is used to encrypt and sign the parameters.

3. The service management and control method based on big data according to claim 1 is characterized in that: The analysis and control process of the data service management and control system includes: When a user accesses the API for the first time, the SET command is first called to set the initialization parameters according to the user's own needs. Then, for each access to the API, the user's traffic control policy configuration is used to check whether the user's access times have reached the access limit within the corresponding time interval.

4. The service management and control method based on big data according to claim 1 is characterized in that: The analysis and control process of the data service management and control system further includes: When the user calls the corresponding API, if the set alarm rule is triggered, the alarm information will be generated and pushed to the KafKa message queue; After the user configures the funnel parameters through the funnel analysis model, a data analysis query task is generated according to the user-configured parameters, and the task request is submitted to the Worker node for execution to obtain the analysis result information.

5. The service management and control method based on big data according to claim 1 is characterized in that: The data transmission and storage of the data service management and control system includes: After receiving the data information that has completed the funnel analysis and processing, it will be transmitted to the corresponding workbench for management personnel to review and analyze; The data information processed in the data service management and control system is stored in the blockchain, which effectively ensures the security of the data information and avoids its loss or tampering.

6. A service management and control system based on big data, characterized by: include: The scheduling management module is used to perform scheduling control management of the data service management and control system; Analysis and processing module, used for operation analysis and processing of data service management and control system; The transmission and storage module is used for transmission and storage of data information.

7. A service management and control system based on big data according to claim 6, characterized in that: The scheduling management module includes: Task scheduling module, used for task scheduling management of data service control system; Gateway management module, used to manage the gateway of the data service management and control system; The processing control module is used to perform operation processing control of the gateway.

8. The big data-based service management and control system according to claim 6, characterized in that: The analysis and processing module comprises: Traffic control module, used for traffic control of data service management and control system; Warning processing module, used for warning analysis and processing of data information; The analysis control module is used to control and manage the funnel analysis model.

9. The service management and control system based on big data according to claim 6, characterized in that: The transmission storage module comprises: An acquisition and transmission module is used to acquire and transmit data information; The data storage module is used to store data information on the blockchain.