Policy data sharing method and device, equipment and medium

Abstract

The embodiment of the invention discloses a policy data sharing method and device, equipment and a medium. The policy data sharing method comprises the steps that policy data acquisition, query and sharing generate a data acquisition micro-service application, a data query micro-service application and a data sharing micro-service application respectively; acquiring policy data from a plurality of preset data sources through the data acquisition micro-service application, and storing the policy data to a database; when a query instruction of specified policy data sent by a user is received, searching the specified policy data in the database through the data query micro-service application; and sharing the specified policy data to the user through the data sharing micro-service application.

Description

Technical Field

[0001] The present specification relates to the field of computer technology, and in particular to a policy data sharing method, device, equipment and medium. Background Art

[0002] With the continuous development of information technology, government departments and enterprises have accumulated a large amount of policy data. This data is of great value to government decision-making, enterprise development and social management. However, since policy data is usually scattered in different data sources, and the formats and standards are also different, how to effectively share policy data has become an urgent problem to be solved.

[0003] Currently, traditional policy data sharing methods often have the following problems:

[0004] Difficulty in data acquisition: Since policy data are scattered across multiple data sources, data acquisition requires docking with each data source, which increases the difficulty and cost of data acquisition.

[0005] Inefficient data query: When querying policy data, it is necessary to search through a large amount of data, which leads to low query efficiency and cannot meet the real-time needs of users. Summary of the invention

[0006] One or more embodiments of this specification provide a policy data sharing method, apparatus, device and medium for solving the technical problems raised by the background technology.

[0007] One or more embodiments of this specification adopt the following technical solutions:

[0008] One or more embodiments of this specification provide a policy data sharing method, the method comprising:

[0009] The acquisition, query and sharing of policy data are generated into data acquisition microservice application, data query microservice application and data sharing microservice application respectively;

[0010] Acquire policy data from multiple preset data sources through the data acquisition microservice application, and store the policy data in a database;

[0011] When receiving a query instruction for designated policy data sent by a user, searching the database for the designated policy data through the data query microservice application;

[0012] The designated policy data is shared with the user through the data sharing microservice application.

[0013] It should be noted that the embodiments of this specification have the following beneficial effects through the above contents:

[0014] Improve data acquisition efficiency: Obtain policy data from multiple preset data sources through the data acquisition microservice application, avoiding the tedious process of connecting with each data source and reducing the difficulty and cost of data acquisition.

[0015] Improve data query efficiency: After storing policy data in the database, the data query microservice application can quickly search for specified policy data in the database, improving query efficiency and meeting users' real-time needs.

[0016] Realize timely data sharing: Data sharing microservice applications can share designated policy data with users in a timely manner, ensuring the timeliness and usage value of the data.

[0017] Solve the problem of data dispersion: Generate microservice applications for the acquisition, query and sharing of policy data respectively, realize the centralized management and unified sharing of policy data, and solve the problem of data being dispersed in different data sources.

[0018] Improve data security: Through microservice applications, policy data can be more finely managed for permissions and access control, improving data security and confidentiality.

[0019] Furthermore, the method further comprises:

[0020] Obtaining tracking information of log points corresponding to the data acquisition microservice application, the data query microservice application, and the data sharing microservice application through a log point module;

[0021] The tracking information of the log embedding point is collected through the information analysis module, and the tracking information of the log embedding point is supervised to obtain a supervision result, which is the monitoring result of the data acquisition microservice application, the data query microservice application and the data sharing microservice application.

[0022] It should be noted that the embodiments of this specification have the following beneficial effects through the above contents:

[0023] Improve the transparency and traceability of data sharing: By obtaining the tracking information of microservice applications through the log tracking module, you can clearly understand the data acquisition, query and sharing process, which improves the transparency of data sharing. At the same time, this tracking information can also be used as a basis for data tracing, which is convenient for investigation and resolution when problems arise.

[0024] Timely discovery and resolution of problems: The information analysis module monitors the tracking information of log points and can monitor the running status of microservice applications in real time. Once an abnormality or problem is found, timely measures can be taken to resolve it, avoiding the expansion of the problem and improving the stability and reliability of the system.

[0025] Optimize system performance: By analyzing the monitoring results, you can understand the performance bottlenecks and resource consumption of microservice applications. Based on this information, you can make targeted optimizations and adjustments to improve system performance and efficiency.

[0026] Furthermore, the obtaining of the tracking information of the log points corresponding to the data acquisition microservice application, the data query microservice application and the data sharing microservice application through the log point module includes:

[0027] Determine the tracking locations of the data acquisition microservice application, data query microservice application, and data sharing microservice application respectively;

[0028] Insert the corresponding log embedding code at the embedding position to obtain the tracking information of the log embedding through the log embedding code.

[0029] It should be noted that the embodiments of this specification have the following beneficial effects through the above contents:

[0030] Accurately locate problems: By determining the tracking location of microservice applications and inserting the corresponding log tracking code, you can accurately obtain the tracking information of each microservice application. This helps to quickly locate the specific link when a problem occurs, improving the efficiency of problem solving.

[0031] Improve debugging efficiency: Developers can quickly understand the execution process and status of microservice applications based on the obtained tracking information, so as to debug and optimize more efficiently.

[0032] Furthermore, the tracking location includes preset key business processes, preset key interface calls and preset key data processing links.

[0033] It should be noted that the embodiments of this specification have the following beneficial effects through the above contents:

[0034] Comprehensive monitoring of business processes: By setting tracking points in preset key business processes, you can comprehensively monitor the execution of the business and understand the execution time, results and other information of each link, which helps to discover bottlenecks and problems in the business process and optimize and improve it.

[0035] Timely discovery of interface problems: Setting tracking points at preset key interface calls can monitor the interface calls in real time, including the number of calls, response time, error information, etc. This helps to timely discover interface performance problems, erroneous calls or abnormal situations, take corresponding measures to repair and optimize, and improve the stability and reliability of the system.

[0036] Accurately locate data processing problems: By setting tracking points in preset key data processing links, you can obtain detailed information about data processing, such as processing time, processing results, data volume, etc. This helps to accurately locate problems in the data processing process, such as data anomalies, processing errors, etc., and make timely repairs and adjustments to ensure the accuracy and integrity of the data.

[0037] Furthermore, the log tracking module adopts bytecode enhancement to transparently and non-invasively access the business party to obtain the tracking information of the log tracking points corresponding to the data acquisition microservice application, the data query microservice application and the data sharing microservice application.

[0038] It should be noted that the embodiments of this specification have the following beneficial effects through the above contents:

[0039] Code-free intrusion: Bytecode enhancement technology eliminates the need for manual tracking in the business code, avoiding direct modification of the business code. This reduces code complexity and maintenance costs, while also reducing the risk of introducing errors.

[0040] Transparency: Business parties do not need to understand or care about the specific implementation details of log tracking. Bytecode enhancement technology automatically inserts log tracking code into the appropriate location during compilation or runtime, which is transparent to business parties. This allows business parties to focus on the development of business logic without worrying about the impact of log tracking.

[0041] Flexibility: Through bytecode enhancement, you can flexibly define and configure the location and information of log tracking points. According to specific needs, you can choose to track points in key business processes, interface calls, or data processing links to obtain the required tracking information.

[0042] Efficiency: Bytecode enhancement technology usually has higher performance because it is optimized during compilation or runtime, reducing additional performance overhead. This helps ensure that log tracking does not have a significant impact on the performance of the business system.

[0043] Maintainability: Since log tracking is automatically implemented through bytecode enhancement, when the configuration of log tracking needs to be modified or updated, there is no need to manually modify the business code. This improves the maintainability and scalability of log tracking.

[0044] Comprehensive tracing information: By performing bytecode enhancement at appropriate locations, comprehensive tracing information can be obtained, including the execution path of the business process, the parameters and results of the interface call, the details of data processing, etc. This helps to gain an in-depth understanding of the operation of the business system, and conduct troubleshooting, performance optimization, and business analysis.

[0045] Quickly locate problems: When a problem occurs, you can quickly locate the location and cause of the problem by viewing the tracking information of the log points. This helps shorten the time of troubleshooting and improve the efficiency of problem solving.

[0046] Furthermore, the method further comprises:

[0047] The supervision results of the data acquisition microservice application, the data query microservice application and the data sharing microservice application are governed by a governance policy module based on a load balancing policy and a fault tolerance policy.

[0048] It should be noted that the embodiments of this specification have the following beneficial effects through the above contents:

[0049] Improve system stability and reliability: Through load balancing and fault tolerance strategies, requests can be distributed to multiple microservice applications to avoid overload or failure of a single microservice application, thereby improving system stability and reliability.

[0050] Optimize resource utilization: The load balancing strategy can dynamically adjust the distribution of requests according to the load of the microservice application, so that resources can be used more reasonably.

[0051] Improve system performance: By allocating requests reasonably, you can reduce the waiting time of requests and improve the response speed and performance of the system.

[0052] Enhance the system's fault tolerance: When a microservice application fails, the fault tolerance strategy can automatically switch to other available microservice applications to ensure the normal operation of the system.

[0053] One or more embodiments of this specification provide a policy data sharing device, including:

[0054] The microservice application generation unit generates a data acquisition microservice application, a data query microservice application, and a data sharing microservice application for the acquisition, query, and sharing of policy data;

[0055] A policy data acquisition unit, which acquires policy data from a plurality of preset data sources through the data acquisition microservice application, and stores the policy data in a database;

[0056] A policy data query unit, when receiving a query instruction for designated policy data sent by a user, searches the database for the designated policy data through the data query microservice application;

[0057] A policy data sharing unit shares the specified policy data with the user through the data sharing microservice application.

[0058] Furthermore, the device also includes:

[0059] A tracking information acquisition unit, which acquires tracking information of log points corresponding to the data acquisition microservice application, the data query microservice application, and the data sharing microservice application through a log point module;

[0060] The monitoring unit collects the tracking information of the log embedding point through the information analysis module, and supervises the tracking information of the log embedding point to obtain a supervision result, which is the monitoring result of the data acquisition microservice application, the data query microservice application and the data sharing microservice application.

[0061] One or more embodiments of this specification provide a policy data sharing device, including:

[0062] at least one processor; and,

[0063] a memory communicatively connected to the at least one processor; wherein,

[0064] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0065] The acquisition, query and sharing of policy data are generated into data acquisition microservice application, data query microservice application and data sharing microservice application respectively;

[0066] Acquire policy data from multiple preset data sources through the data acquisition microservice application, and store the policy data in a database;

[0067] When receiving a query instruction for designated policy data sent by a user, searching the database for the designated policy data through the data query microservice application;

[0068] The designated policy data is shared with the user through the data sharing microservice application.

[0069] One or more embodiments of this specification provide a non-volatile computer storage medium storing computer executable instructions, which can achieve the following when executed by a computer:

[0070] The acquisition, query and sharing of policy data are generated into data acquisition microservice application, data query microservice application and data sharing microservice application respectively;

[0071] Acquire policy data from multiple preset data sources through the data acquisition microservice application, and store the policy data in a database;

[0072] When receiving a query instruction for designated policy data sent by a user, searching the database for the designated policy data through the data query microservice application;

[0073] The designated policy data is shared with the user through the data sharing microservice application.

[0074] At least one of the above technical solutions adopted in the embodiments of this specification can achieve the following beneficial effects:

[0075] Improve data acquisition efficiency: Obtain policy data from multiple preset data sources through the data acquisition microservice application, avoiding the tedious process of connecting with each data source and reducing the difficulty and cost of data acquisition.

[0076] Improve data query efficiency: After storing policy data in the database, the data query microservice application can quickly search for specified policy data in the database, improving query efficiency and meeting users' real-time needs.

[0077] Realize timely data sharing: Data sharing microservice applications can share designated policy data with users in a timely manner, ensuring the timeliness and usage value of the data.

[0078] Solve the problem of data dispersion: Generate microservice applications for the acquisition, query and sharing of policy data respectively, realize the centralized management and unified sharing of policy data, and solve the problem of data being dispersed in different data sources.

[0079] Improve data security: Through microservice applications, policy data can be more finely managed for permissions and access control, improving data security and confidentiality. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] In order to more clearly illustrate the technical solutions in the embodiments of this specification or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art description. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. In the drawings:

[0081] Figure 1 A flowchart of a policy data sharing method provided for one or more embodiments of this specification;

[0082] Figure 2 A schematic diagram of the structure of a policy data sharing device provided for one or more embodiments of this specification;

[0083] Figure 3A schematic diagram of the structure of a policy data sharing device provided for one or more embodiments of this specification. DETAILED DESCRIPTION

[0084] The embodiments of this specification provide a policy data sharing method, device, equipment and medium.

[0085] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0086] Figure 1 This is a flow chart of a policy data sharing method provided in one or more embodiments of this specification, and the flow can be executed by a policy data sharing system. Certain input parameters or intermediate results in the flow allow manual intervention and adjustment to help improve accuracy.

[0087] The method steps of the embodiment of this specification are as follows:

[0088] S102, the acquisition, query and sharing of policy data are respectively generated into a data acquisition microservice application, a data query microservice application and a data sharing microservice application.

[0089] In the examples of this specification, the above content can be implemented by the following specific implementation scheme:

[0090] Data acquisition microservice: Develop an interface or script to obtain policy data from the data source. Implement the data extraction, transformation, and loading (ETL) process to organize the data into a format suitable for querying and sharing. Provide a data update mechanism to ensure the timeliness and accuracy of the data.

[0091] Data query microservice: Build query interfaces to support user query requests for policy data. Design efficient query algorithms and indexes to improve query performance. Process the return and display of query results.

[0092] Data Sharing Microservices: Develop interfaces to support sharing requests for policy data from external systems or users. Ensure data security and permission control, allowing only authorized users or systems to access and share data. Handle data transmission and format conversion to meet the needs of different systems.

[0093] Microservice deployment: Deploy each microservice to an independent container or server to achieve independent operation and expansion of microservices. Use container orchestration tools (such as Kubernetes) for management and scheduling.

[0094] Integration and testing: Perform integration tests between microservices to ensure that they can interact and work together correctly. Perform system-level testing to verify the functionality and performance of the entire system.

[0095] Monitoring and operation and maintenance: Establish a monitoring mechanism to monitor the operating status, performance indicators and error logs of microservices in real time. Formulate operation and maintenance strategies, including backup, recovery, upgrade and other operations.

[0096] S104: Obtain policy data from multiple preset data sources through the data acquisition microservice application, and store the policy data in a database.

[0097] In the examples of this specification, the above content can be implemented by the following specific implementation scheme:

[0098] Data source configuration: Determine which preset data sources to obtain policy data from and configure the corresponding data acquisition interface or API.

[0099] Data acquisition microservice application development: Use appropriate programming languages ​​and frameworks to develop data acquisition microservice applications. Implement data acquisition logic to acquire policy data by calling the data source interface or API. Handle data parsing and conversion to convert acquired data into a unified format.

[0100] Database connection: Establish a connection with the database to ensure that data can be stored successfully.

[0101] Data storage: Store the acquired policy data in a database. You can use a relational database (such as MySQL, Oracle) or a non-relational database (such as MongoDB, Cassandra), depending on the characteristics and requirements of the data. Design a suitable database table structure to store the various fields and attributes of the policy data.

[0102] Data update strategy: Determine how often and how data is updated to ensure that policy data in the database is always up to date.

[0103] S106, when receiving a query instruction for designated policy data sent by a user, searching the database for the designated policy data through the data query microservice application.

[0104] In the examples of this specification, the above content can be implemented by the following specific implementation scheme:

[0105] Data query microservice application development: Use appropriate programming languages ​​and frameworks to develop data query microservice applications. Implement query logic to find specified policy data by connecting to the database and executing query statements. Handle the return and display of query results.

[0106] Database connection: Establish a connection between the data query microservice application and the database to ensure that the policy data in the database can be accessed.

[0107] Query instruction processing: Receive the query instructions sent by the user and pass them to the data query microservice application. Parse the query instructions and extract relevant information of the specified policy data.

[0108] Database query: Execute the corresponding query operation in the database according to the parsed query instructions. Use appropriate query statements and indexes to improve query performance.

[0109] Query result return: Get the query results and return them to the user interface. The query results can be displayed to the user in a table, chart or other form.

[0110] Error handling and logging: Handle possible error conditions that may occur during the query process and record relevant error logs for troubleshooting and monitoring.

[0111] S108: Share the designated policy data with the user through the data sharing microservice application.

[0112] In the examples of this specification, the above content can be implemented by the following specific implementation scheme:

[0113] Data access permission management: Set corresponding data access permissions for different users or user groups to ensure that users can only share the data they are authorized to access.

[0114] Data sharing interface design: Design the data sharing interface and define the methods and parameters through which users can request to share specified policy data.

[0115] Data extraction and preparation: Based on user requests, extract designated policy data from databases or other data sources, and perform necessary data processing and conversion to meet sharing requirements.

[0116] Data transmission and encryption: Select the appropriate data transmission method to transmit the specified policy data to the user. Encryption technology can be used to protect the security of data during transmission.

[0117] User feedback and notification: Provide a user feedback mechanism to allow users to confirm whether data is shared successfully, and notify users of the status and results of the sharing in a timely manner.

[0118] Monitoring and logging: Establish a monitoring mechanism to monitor the process and results of data sharing in real time. At the same time, record relevant log information for troubleshooting and auditing.

[0119] Furthermore, the method also includes: obtaining the tracking information of the log tracking points corresponding to the data acquisition microservice application, the data query microservice application and the data sharing microservice application through the log tracking point module; collecting the tracking information of the log tracking points through the information analysis module, and supervising the tracking information of the log tracking points to obtain the supervision results, which are the monitoring results of the data acquisition microservice application, the data query microservice application and the data sharing microservice application.

[0120] In the examples of this specification, the above content can be implemented by the following specific implementation scheme:

[0121] Design and implementation of log tracking module: Determine the locations and events where log tracking is required in data acquisition microservice applications, data query microservice applications, and data sharing microservice applications. Use appropriate technologies and frameworks to add log tracking statements in application code to record key tracking information, such as request parameters, response results, processing time, etc.

[0122] Information analysis module design and implementation: Develop an information analysis module to collect and process the tracking information of log points. This module can use log collection tools (such as Logstash, Fluentd, etc.) to collect log data and store it in a suitable storage medium (such as Elasticsearch, database, etc.).

[0123] Governance rule definition: Determine governance rules and metrics to evaluate the performance and behavior of data acquisition microservice applications, data query microservice applications, and data sharing microservice applications. These rules can include request success rate, response time threshold, data accuracy, etc.

[0124] Real-time monitoring and alerting: Establish a real-time monitoring mechanism to monitor the performance and behavior of applications in real time by querying stored log data. When the monitoring results exceed the preset threshold or rules, an alert notification is triggered so that timely measures can be taken.

[0125] It should be noted that the embodiments of this specification have the following beneficial effects through the above contents:

[0126] Improve the transparency and traceability of data sharing: By obtaining the tracking information of microservice applications through the log tracking module, you can clearly understand the data acquisition, query and sharing process, which improves the transparency of data sharing. At the same time, this tracking information can also be used as a basis for data tracing, which is convenient for investigation and resolution when problems arise.

[0127] Timely discovery and resolution of problems: The information analysis module monitors the tracking information of log points and can monitor the running status of microservice applications in real time. Once an abnormality or problem is found, timely measures can be taken to resolve it, avoiding the expansion of the problem and improving the stability and reliability of the system.

[0128] Optimize system performance: By analyzing the monitoring results, you can understand the performance bottlenecks and resource consumption of microservice applications. Based on this information, you can make targeted optimizations and adjustments to improve system performance and efficiency.

[0129] Furthermore, when the log burying point module obtains the tracking information of the log burying points corresponding to the data acquisition microservice application, the data query microservice application and the data sharing microservice application, the burying point locations of the data acquisition microservice application, the data query microservice application and the data sharing microservice application are determined respectively; and the corresponding log burying point code is inserted into the burying point location to obtain the tracking information of the log burying point through the log burying point code.

[0130] In the examples of this specification, the above content can be implemented by the following specific implementation scheme:

[0131] Determine the tracking location: Work with the development teams of the data acquisition microservice application, data query microservice application, and data sharing microservice application to understand the application architecture and functional flow. According to business needs and monitoring goals, determine the key locations where tracking is required, such as: the entry and exit points of data acquisition. The request and response processing points of data query. The initiation and completion points of data sharing.

[0132] Insert log tracking code: Use the corresponding programming language and framework to insert the log tracking code at the determined tracking location. The log tracking code can record the following information:

[0133] Timestamp: records the time when the event occurred.

[0134] Operation type: For example, data acquisition, query, or sharing.

[0135] Related parameters: including request parameters, data identifiers, etc.

[0136] Result status: For example, success or failure.

[0137] Other relevant information: More context information can be recorded based on specific needs.

[0138] Configure logging: Configure the logging method and storage location. You can choose to log to a file, database, or other log management system. Ensure that the logging configuration meets the needs of monitoring and analysis, such as setting appropriate log levels, log rotation strategies, etc.

[0139] Testing and verification: Test in the development environment to ensure the correctness and stability of the logging code. Verify that the log records contain the expected tracking information and accurately reflect the behavior of the application.

[0140] Deployment and monitoring: Deploy the modified application to the production environment. Establish a monitoring mechanism to check log records in real time or regularly to ensure the normal operation of the application and the accuracy of the data.

[0141] Data analysis and utilization: Use log analysis tools or custom scripts to analyze the collected log data. You can extract valuable information, such as the frequency and success rate of statistical data acquisition, query, and sharing. Analyze the time and performance bottlenecks of data processing. Detect abnormalities or errors and take timely measures. Provide a basis for optimizing application performance and improving business processes.

[0142] It should be noted that the embodiments of this specification have the following beneficial effects through the above contents:

[0143] Accurately locate problems: By determining the tracking location of microservice applications and inserting the corresponding log tracking code, you can accurately obtain the tracking information of each microservice application. This helps to quickly locate the specific link when a problem occurs, improving the efficiency of problem solving.

[0144] Improve debugging efficiency: Developers can quickly understand the execution process and status of microservice applications based on the obtained tracking information, so as to debug and optimize more efficiently.

[0145] Furthermore, the tracking location includes preset key business processes, preset key interface calls and preset key data processing links.

[0146] It should be noted that the embodiments of this specification have the following beneficial effects through the above contents:

[0147] Comprehensive monitoring of business processes: By setting tracking points in preset key business processes, you can comprehensively monitor the execution of the business and understand the execution time, results and other information of each link, which helps to discover bottlenecks and problems in the business process and optimize and improve it.

[0148] Timely discovery of interface problems: Setting tracking points at preset key interface calls can monitor the interface calls in real time, including the number of calls, response time, error information, etc. This helps to timely discover interface performance problems, erroneous calls or abnormal situations, take corresponding measures to repair and optimize, and improve the stability and reliability of the system.

[0149] Accurately locate data processing problems: By setting tracking points in preset key data processing links, you can obtain detailed information about data processing, such as processing time, processing results, data volume, etc. This helps to accurately locate problems in the data processing process, such as data anomalies, processing errors, etc., and make timely repairs and adjustments to ensure the accuracy and integrity of the data.

[0150] Furthermore, the log tracking module adopts bytecode enhancement to transparently and non-invasively access the business party to obtain the tracking information of the log tracking points corresponding to the data acquisition microservice application, the data query microservice application and the data sharing microservice application.

[0151] It should be noted that the embodiments of this specification have the following beneficial effects through the above contents:

[0152] Code-free intrusion: Bytecode enhancement technology eliminates the need for manual tracking in the business code, avoiding direct modification of the business code. This reduces code complexity and maintenance costs, while also reducing the risk of introducing errors.

[0153] Transparency: Business parties do not need to understand or care about the specific implementation details of log tracking. Bytecode enhancement technology automatically inserts log tracking code into the appropriate location during compilation or runtime, which is transparent to business parties. This allows business parties to focus on the development of business logic without worrying about the impact of log tracking.

[0154] Flexibility: Through bytecode enhancement, you can flexibly define and configure the location and information of log tracking points. According to specific needs, you can choose to track points in key business processes, interface calls, or data processing links to obtain the required tracking information.

[0155] Efficiency: Bytecode enhancement technology usually has higher performance because it is optimized during compilation or runtime, reducing additional performance overhead. This helps ensure that log tracking does not have a significant impact on the performance of the business system.

[0156] Maintainability: Since log tracking is automatically implemented through bytecode enhancement, when the configuration of log tracking needs to be modified or updated, there is no need to manually modify the business code. This improves the maintainability and scalability of log tracking.

[0157] Comprehensive tracing information: By performing bytecode enhancement at appropriate locations, comprehensive tracing information can be obtained, including the execution path of the business process, the parameters and results of the interface call, the details of data processing, etc. This helps to gain an in-depth understanding of the operation of the business system, and conduct troubleshooting, performance optimization, and business analysis.

[0158] Quickly locate problems: When a problem occurs, you can quickly locate the location and cause of the problem by viewing the tracking information of the log points. This helps shorten the time of troubleshooting and improve the efficiency of problem solving.

[0159] Furthermore, the supervision results of the data acquisition microservice application, the data query microservice application and the data sharing microservice application can be governed by a governance policy module based on a load balancing policy and a fault tolerance policy.

[0160] It should be noted that the embodiments of this specification have the following beneficial effects through the above contents:

[0161] Improve system stability and reliability: Through load balancing and fault tolerance strategies, requests can be distributed to multiple microservice applications to avoid overload or failure of a single microservice application, thereby improving system stability and reliability.

[0162] Optimize resource utilization: The load balancing strategy can dynamically adjust the distribution of requests according to the load of the microservice application, so that resources can be used more reasonably.

[0163] Improve system performance: By allocating requests reasonably, you can reduce the waiting time of requests and improve the response speed and performance of the system.

[0164] Enhance the system's fault tolerance: When a microservice application fails, the fault tolerance strategy can automatically switch to other available microservice applications to ensure the normal operation of the system.

[0165] It should be noted that at this stage, there are more and more platforms developed based on microservice architecture. With the development of business, the number of services is increasing. After the application service is transformed, it brings great challenges to system operation and maintenance. Developers complete communication in different ways, making the discovery between services, tracking of call links and quality issues quite difficult. How to solve the challenges brought by microservice architecture will be the development trend of future architecture. Each additional service will increase the difficulty of operation. The increase in the number of services means that the complexity of communication calls will increase. How to reasonably monitor the internal operation of each service will be the number one problem in the development of microservices. For this reason, this paper proposes a design method for a service monitoring system in a microservice architecture, including: 1. Design a tracking agent to complete the tracking data tracking point; 2. Design an information analysis module to complete the collection and analysis of the tracking point data; 3. Design a UI front-end display module to complete the visual display of the call link; 4. Design a configuration center module to uniformly manage system configuration files; 5. Design a service governance module for targeted governance.

[0166] Microservices: Microservices describe an architectural style. A large and complex application consists of multiple microservice modules. Each microservice module in the system can be deployed and developed independently, and different microservices are loosely coupled. Each microservice focuses on completing only one task, and each microservice represents a small business capability. Microservices can communicate with each other through HTTP or RPC calls. In the microservice architecture, you only need to add corresponding functional modules to meet the increase in demand without affecting the performance of the overall application.

[0167] Link tracking system: For ordinary systems or services, logging is generally used to track points, and some log analysis tools are used to locate and analyze problems. As the business becomes more and more complex, enterprise applications have also entered the stage of distributed services. Traditional monitoring methods cannot meet the needs of call tracking and troubleshooting. In the microservice architecture, services are usually divided by business. Using Restful API calls and exposing interfaces to the outside world, it may require many services to collaborate to complete the function of an interface call. If any service module on the link has a problem, the interface call will fail.

[0168] Link tracking means that from the beginning to the end of a task, all the systems called and the time consumed (time span) during the period can be fully recorded, and what is presented is each system call involved and the call link and time span within the system.

[0169] The link tracking system is similar to many monitoring systems and is also divided into basic data collection + data storage + front-end display.

[0170] The purpose of this system is to design and implement a service monitoring system in applications or platforms developed with microservice architecture. The system collects logs of each microservice during operation and stores them in a centralized service. When a problem occurs in the platform or application, the problem can be quickly located. At the same time, some operating indicators within the application and some system-wide indicators are also collected. This system is different from the traditional log system. It aggregates the collected data. Through link statistics, it can clearly know where the bottleneck of the entire platform or application is, where to expand capacity or plan for some potential problems. With this system, developers can focus more on business-level development, and integrate the problems at the service governance level into the system, completing the decoupling between business and weak business. Improve the stability of the entire system and enhance the availability and fault tolerance of the system.

[0171] The system adopts a modular concept and divides the entire system into: log point module, information analysis module, and WebUI module. The information analysis module includes the log collection (Collector) module and the data processing module. These modules together constitute the service monitoring system. Each functional module performs its own duties and complements each other to complete the service monitoring function.

[0172] The system data refers to the link analysis information data collected by the probe (Trace Agent). The data link information obtained by the probe is sent to the Kafka message queue. The data analysis module uses Spark to build a data processing flow with a total of four operators. Each operator processes the data and enters it into the Collector module, which collects and stores the data in ES. The main function of the UI module is to display link data and Kibana visualization data. In addition, the Warning Push alarm function has been developed in the WebUI module. This function mainly formulates some rules and personnel matching, processes the data read from ES one by one to complete the rule matching, and pushes services with abnormal status to the responsible personnel of the business line.

[0173] Log tracking module:

[0174] Trace Agent mainly uses bytecode enhancement to transparently and non-invasively access the business side to obtain the tracking information of log points. Trace Agent draws on the idea of ​​Service Mesh. Its function is the same as Sidecar in Service Mesh. All microservice deployments require an Agent with the same life cycle and service to complete the information tracking and monitoring of distributed messages.

[0175] Information analysis module:

[0176] The information analysis module is a module that integrates collection, storage, and analysis. Its main function is to collect the tracking information collected by the Agent, analyze and process it through the data processing module, and finally store it in ES for the WebUI module to call. This module is the core of the data flow in the entire system. The tracking data collected by the Trace Agent will be analyzed and processed in the information analysis module.

[0177] The open source ELK architecture is used to efficiently implement collection and aggregation. The message queue Kafka is used as the cache band for the collection node and the aggregation processing node. The built-in reliability and data persistence mechanism of Kafka is used to ensure that the tracking information data will not be lost to a certain extent, and can be reliably passed to the data processing module for processing.

[0178] WebUI Module:

[0179] The overall design pattern of this module adopts Controller layer (exposed interface), Service layer (business logic related), Dao layer (database connection related). The entire module uses the MVC (View-Model-Control) mode to make the development hierarchy clearer and complete the decoupling function.

[0180] The view layer mainly completes Web page design, and uses front-end technologies (HTML, CSS, JS, Vuejs, etc.) to implement the system's UI interface, which can display user-friendly pages more intuitively and clearly. Users can view application overviews, link information, log information, etc.

[0181] Configuration center module:

[0182] There may be many nodes in the cluster running the same task. This task may have some parameter configurations that may change frequently. The requirement is that when the configuration parameters change, they can be quickly synchronized to each node. The system designs a distributed configuration center based on zookeeper, and establishes a root node in zookeeper, such as / Config. Represents a configuration file. The information in the configuration file is stored as a child node of the root node. For example, if the configuration successRate<80 is displayed in zookeeper as / config / successRate, and the content of the node is <80. Then let all application machines that use the configuration information integrate zookeeper and monitor the status of / Config. Once the configuration information changes, each machine will receive a notification from zookeeper, and then obtain the new configuration information from zookeeper and apply it to the system.

[0183] Governance policy module:

[0184] In order to carry out targeted governance of abnormal service problems found in the service monitoring system, a governance strategy module is designed. This module uses the relatively mature SpringCloud components at this stage to govern services, including load balancing strategies and fault tolerance strategies, and implements load balancing, service degradation, circuit breakers, etc. It embeds open source components into services and completes governance through their exposed APIs.

[0185] Design a transparent, non-intrusive code tracking information embedding probe to allow the business side to collect tracking information without being aware of it. It has high scalability and fault tolerance, little performance loss on the service, and stable performance.

[0186] Design a reasonable information analysis module that can process, analyze and store a large amount of tracking data, and combine Kafka, ES, and Kibana to complete storage and analysis. The data processing module can perform streaming and controllable analysis and processing of tracking data.

[0187] Design a displayable WebUI module that allows users to use the system quickly and easily, and intuitively display application profiles and link-related data. Design rule filtering to push service alarm information that meets the requirements to application developers to handle the problem in a timely manner.

[0188] Figure 2 A structural diagram of a policy data sharing device provided for one or more embodiments of this specification includes: a microservice application generation unit 202 , a policy data acquisition unit 204 , a policy data query unit 206 and a policy data sharing unit 208 .

[0189] The microservice application generation unit 202 generates a data acquisition microservice application, a data query microservice application and a data sharing microservice application for acquisition, query and sharing of policy data respectively;

[0190] A policy data acquisition unit 204 acquires policy data from a plurality of preset data sources through the data acquisition microservice application, and stores the policy data in a database;

[0191] The policy data query unit 206 searches the database for the specified policy data through the data query microservice application when receiving a query instruction for the specified policy data sent by the user;

[0192] The policy data sharing unit 208 shares the specified policy data with the user through the data sharing microservice application.

[0193] Furthermore, the device also includes:

[0194] A tracking information acquisition unit, which acquires tracking information of log points corresponding to the data acquisition microservice application, the data query microservice application, and the data sharing microservice application through a log point module;

[0195] The monitoring unit collects the tracking information of the log embedding point through the information analysis module, and supervises the tracking information of the log embedding point to obtain a supervision result, which is the monitoring result of the data acquisition microservice application, the data query microservice application and the data sharing microservice application.

[0196] Figure 3 A schematic diagram of a policy data sharing device provided for one or more embodiments of this specification includes:

[0197] at least one processor; and,

[0198] a memory communicatively connected to the at least one processor; wherein,

[0199] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0200] The acquisition, query and sharing of policy data are generated into data acquisition microservice application, data query microservice application and data sharing microservice application respectively;

[0201] Acquire policy data from multiple preset data sources through the data acquisition microservice application, and store the policy data in a database;

[0202] When receiving a query instruction for designated policy data sent by a user, searching the database for the designated policy data through the data query microservice application;

[0203] The designated policy data is shared with the user through the data sharing microservice application.

[0204] One or more embodiments of this specification provide a non-volatile computer storage medium storing computer executable instructions, which can achieve the following when executed by a computer:

[0205] The acquisition, query and sharing of policy data are generated into data acquisition microservice application, data query microservice application and data sharing microservice application respectively;

[0206] Acquire policy data from multiple preset data sources through the data acquisition microservice application, and store the policy data in a database;

[0207] When receiving a query instruction for designated policy data sent by a user, searching the database for the designated policy data through the data query microservice application;

[0208] The designated policy data is shared with the user through the data sharing microservice application.

[0209] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

Claims

1. A policy data sharing method, characterized in that: The method comprises: The acquisition, query and sharing of policy data are generated into data acquisition microservice application, data query microservice application and data sharing microservice application respectively; Acquire policy data from multiple preset data sources through the data acquisition microservice application, and store the policy data in a database; When receiving a query instruction for designated policy data sent by a user, searching the database for the designated policy data through the data query microservice application; The designated policy data is shared with the user through the data sharing microservice application.

2. The method according to claim 1, characterized in that The method further comprises: Obtaining tracking information of log points corresponding to the data acquisition microservice application, the data query microservice application, and the data sharing microservice application through a log point module; The tracking information of the log embedding point is collected through the information analysis module, and the tracking information of the log embedding point is supervised to obtain a supervision result, which is the monitoring result of the data acquisition microservice application, the data query microservice application and the data sharing microservice application.

3. The method according to claim 2, characterized in that The obtaining, through the log tracking module, the tracking information of the log tracking points corresponding to the data acquisition microservice application, the data query microservice application, and the data sharing microservice application includes: Determine the tracking locations of the data acquisition microservice application, data query microservice application, and data sharing microservice application respectively; Insert the corresponding log embedding code at the embedding position to obtain the tracking information of the log embedding through the log embedding code.

4. The method according to claim 3, characterized in that: The tracking point locations include preset key business processes, preset key interface calls and preset key data processing links.

5. The method according to claim 2, characterized in that: The log tracking module adopts bytecode enhancement to transparently and non-invasively access the business party to obtain the tracking information of the log tracking points corresponding to the data acquisition microservice application, the data query microservice application and the data sharing microservice application.

6. The method according to claim 2, characterized in that The method further comprises: The supervision results of the data acquisition microservice application, the data query microservice application and the data sharing microservice application are governed by a governance policy module based on a load balancing policy and a fault tolerance policy.

7. A policy data sharing device, characterized in that: include: The microservice application generation unit generates a data acquisition microservice application, a data query microservice application, and a data sharing microservice application for the acquisition, query, and sharing of policy data; A policy data acquisition unit, which acquires policy data from a plurality of preset data sources through the data acquisition microservice application, and stores the policy data in a database; A policy data query unit, when receiving a query instruction for designated policy data sent by a user, searches the database for the designated policy data through the data query microservice application; A policy data sharing unit shares the specified policy data with the user through the data sharing microservice application.

8. The device according to claim 7, characterized in that The device also includes: A tracking information acquisition unit, which acquires tracking information of log points corresponding to the data acquisition microservice application, the data query microservice application, and the data sharing microservice application through a log point module; The monitoring unit collects the tracking information of the log embedding point through the information analysis module, and supervises the tracking information of the log embedding point to obtain a supervision result, which is the monitoring result of the data acquisition microservice application, the data query microservice application and the data sharing microservice application.

9. A policy data sharing device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to: The acquisition, query and sharing of policy data are generated into data acquisition microservice application, data query microservice application and data sharing microservice application respectively; Acquire policy data from multiple preset data sources through the data acquisition microservice application, and store the policy data in a database; When receiving a query instruction for designated policy data sent by a user, searching the database for the designated policy data through the data query microservice application; The designated policy data is shared with the user through the data sharing microservice application.

10. A non-volatile computer storage medium, characterized in that: Computer executable instructions are stored, and when the computer executable instructions are executed by a computer, the following can be achieved: The acquisition, query and sharing of policy data are generated into data acquisition microservice application, data query microservice application and data sharing microservice application respectively; Acquire policy data from multiple preset data sources through the data acquisition microservice application, and store the policy data in a database; When receiving a query instruction for designated policy data sent by a user, searching the database for the designated policy data through the data query microservice application; The designated policy data is shared with the user through the data sharing microservice application.

Images