Service information management method and device

Abstract

The invention discloses a business information management method and device. The method comprises the steps that a pre-configured section collection plug-in is used for collecting record information of a business system calling information collection application program interface, the record information is written into a memory queue, and the type of the memory queue is a consumption blocking queue; a pre-configured first thread is utilized to read the record information in the memory queue, the record information is written into a list cache corresponding to the first thread, when the list cache reaches a preset condition, all the record information in the list cache is written into a log file of a local disk, and the preset condition comprises at least one of a data volume threshold value and a waiting time threshold value; and periodically reading the incremental log files in the local disk by utilizing a pre-configured second thread, and writing all the incremental log files into the target database in batches. According to the application, the technical problem that related service information is difficult to efficiently obtain when user services are verified in a complaint management scene is solved.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and more specifically, to a method and apparatus for managing service information. Background Art

[0002] In today's highly digital and networked environment, computers and data processing systems carry a vast amount of user service requests. Especially in high-concurrency scenarios, such as peak periods of e-commerce activities and large-scale social network events, the amount of data that the system needs to process increases exponentially. Against this background, how to effectively collect and store key service information under resource constraints has become a key issue in improving service quality and ensuring user experience.

[0003] Existing service information collection methods and systems often have serious problems of information acquisition delay and accuracy when facing user complaints. When verifying the specific reasons for user service failures, complainants are limited by the complexity of service logic and the concealment of rule details, and it is difficult to directly obtain comprehensive and specific information. Usually, they need to submit user information to the service provider or developer and wait for confirmation from the other party to know the problem, which is often time-consuming and inefficient. In addition, in high-concurrency situations, due to limitations of external resources such as memory and hard disks, traditional information collection methods may result in incomplete log records. Even when the system load is too high, the log recording function is restricted, affecting the complete collection of key information. More specifically, in high-concurrency scenarios, the log records of service systems often encounter the problem of memory queue overflow, which not only increases the burden on the system but also may cause important service exception information to be lost, thus affecting subsequent problem location and solution.

[0004] To address the above problems, no effective solution has been proposed yet. Summary of the Invention

[0005] Embodiments of this application provide a method and apparatus for managing service information to at least solve the technical problem of difficultly obtaining relevant service information efficiently when verifying user services in a complaint management scenario.

[0006] According to one aspect of the embodiments of the present application, a method for managing service information is provided, including: collecting record information of an application programming interface for collecting service system call information by using a pre-configured aspect collection plug-in, and writing the record information into a memory queue, where the type of the memory queue is a consumption blocking queue; reading the record information in the memory queue by using a pre-configured first thread, and writing the read record information into a list cache corresponding to the first thread. When the list cache meets a preset condition, writing all the record information in the list cache into a log file on the local disk, where the preset condition includes at least one of the following: a data volume threshold, a waiting time threshold; periodically reading incremental log files on the local disk by using a pre-configured second thread, and batch writing all the incremental log files into a target database.

[0007] Optionally, before collecting record information of the application programming interface for collecting service system call information by using the pre-configured aspect collection plug-in, the above method further includes: configuring a general application programming interface for collecting service information and a deduplication interface for interacting with the service system, and forming an information collection application programming interface based on the general application programming interface and the deduplication interface; configuring an aspect point and an initialization method corresponding to the aspect collection plug-in, where the initialization method is singleton startup; configuring first configuration parameters corresponding to the first thread, where at least the first configuration parameters include: a queue consumption method, a preset condition; configuring second configuration parameters corresponding to the second thread, where at least the second configuration parameters include: batch writing parameters.

[0008] Optionally, collecting record information of the application programming interface for collecting service system call information by using the pre-configured aspect collection plug-in, and writing the record information into the memory queue includes: starting the aspect collection plug-in as a singleton based on the double-checked locking mechanism; initializing the memory queue; collecting record information of the application programming interface for collecting service system call information by using the aspect collection plug-in, and writing the record information into the initialized memory queue.

[0009] Optionally, writing the record information into the memory queue includes: reading the record information in the memory queue by using the first thread through the poll method; performing deduplication processing on the read record information; writing the deduplicated record information into the list cache corresponding to the first thread.

[0010] Optionally, when the list cache meets the preset condition, writing all the record information in the list cache into the log file on the local disk includes: when the data volume in the list cache reaches the data volume threshold or the waiting time corresponding to the list cache reaches the waiting time threshold, creating a log file on the local disk, writing all the record information in the list cache into the log file, and clearing the list cache.

[0011] Optionally, a pre-configured second thread is used to periodically read the incremental log files in the local disk and batch write all the incremental log files into the target database, including: periodically comparing the log files in the local disk and the target database by using the second thread to determine the incremental log files in the local disk; grouping the incremental log files according to the pre-configured batch write parameters; and batch writing the grouped incremental log files into the target database based on the remote procedure call interface.

[0012] According to another aspect of the embodiments of the present application, a management device for service information is further provided, including: a collection module, configured to collect the record information of the application program interface for collecting service system call information by using a pre-configured aspect collection plug-in and write the record information into a memory queue, where the type of the memory queue is a consumption blocking queue; a first storage module, configured to read the record information in the memory queue by using a pre-configured first thread and write the read record information into a list cache corresponding to the first thread, and when the list cache meets a preset condition, write all the record information in the list cache into the log file in the local disk, where the preset condition includes at least one of the following: a data volume threshold, a waiting time threshold; a second storage module, configured to periodically read the incremental log files in the local disk by using a pre-configured second thread and batch write all the incremental log files into the target database.

[0013] According to another aspect of the embodiments of the present application, a computer program product is further provided, including: a computer program, where when the computer program is executed by a processor, the management method of the above-mentioned service information is implemented.

[0014] According to another aspect of the embodiments of the present application, an electronic device is further provided, including: a memory and a processor, where a computer program is stored in the memory, and the processor is configured to execute the management method of the above-mentioned service information through the computer program.

[0015] In the embodiments of the present application, a pre-configured aspect collection plug-in is used to collect the record information of the service system in a non-intrusive manner. In an environment with limited resources, a dual-thread mechanism is adopted, the local disk file is used as a data buffer, and a multi-level storage strategy from the memory queue to the local file and then to the database is used. Finally, the incremental log files are written into the target database, so as to realize the efficient and accurate collection and storage of key service information in a high-concurrency scenario, and further solve the technical problem that it is difficult to efficiently obtain relevant service information when verifying user services in the complaint management scenario. Description of the Drawings

[0016] The accompanying drawings described herein are used to provide a further understanding of the present application, and constitute a part of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present application, and do not constitute an improper limitation of the present application. In the drawings:

[0017] Figure 1 It is a schematic flowchart of a method for managing an optional service information according to an embodiment of the present application;

[0018] Figure 2 It is a schematic structural diagram of a device for managing an optional service information according to an embodiment of the present application;

[0019] Figure 3 It is a schematic structural diagram of an electronic device according to an embodiment of the present application. Detailed implementation manners

[0020] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0021] It should be noted that the terms "first", "second", etc. in the specification, claims and drawings of the present application are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented in an order different from those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0022] In order to better understand the embodiments of the present application, some nouns or terms that appear in the description process of the embodiments of the present application are translated and explained as follows:

[0023] Aspect: An aspect is one of the core concepts in Aspect - Oriented Programming (AOP), which is used to describe the modularization of cross - cutting concerns such as logging, security checks, transaction management, etc. In this technical solution, the aspect is used to collect key information during the execution of business operations without modifying the business logic code, realizing the separation of business logic and logging, and improving the maintainability and extensibility of the code.

[0024] Aspect collection plugins: Aspect collection plugins are very common in AOP. They are used to capture cross - cutting concerns such as logging, performance monitoring, transaction management, etc. without invading the business logic code. Common aspect collection plugins and technologies include: Spring AOP, AspectJ, Log4j Interceptor, Zipkin. Spring AOP: The Spring framework is one of the most commonly used frameworks in Java development. Its AOP module provides powerful aspect - oriented programming support, allowing developers to define pointcuts and aspects, and add functions such as logging and security checks without modifying the business code. Spring AOP uses the proxy pattern to implement aspects and can be very flexibly applied to various business scenarios; AspectJ: AspectJ is a complete implementation of aspect - oriented programming and can be seamlessly integrated with Java code. It provides more powerful aspect - oriented programming capabilities than Spring AOP, including inserting cross - cutting concern code at compile - time or runtime. AspectJ: Supports more complex pointcut expressions and can more precisely control the triggering conditions of aspects; Log4jInterceptor: A popular logging framework that provides the concept of interceptors and can be used to automatically insert log records at specific points in the code. This mechanism is similar to aspects but is more focused on the logging function and can be used to record information such as method calls and exception handling; Zipkin: Another distributed tracing system, similar to Jaeger, which provides rich application programming interfaces for tracking requests across services. Zipkin aspects can help developers collect and analyze the execution path and performance data of requests without modifying the business logic.

[0025] Double-checked locking mechanism: A technology for optimizing the implementation of the singleton pattern, mainly aiming to reduce lock contention in a multi-threaded environment, thereby improving performance. In languages such as Java, when creating a singleton object, it is usually necessary to ensure thread safety, that is, when multiple threads attempt to create an object simultaneously, only one thread is allowed to successfully create it, and other threads are blocked until the object is created. The double-checked locking mechanism achieves this purpose through two checks (double-check): The first check is to check whether the object has been created before acquiring the lock. If the object has been created, then directly return the object, avoiding unnecessary lock operations; if the object has not been created in the first check, then truly acquire the lock and check again whether the object has been created by other threads. If the object is still not created, then create and initialize the object.

[0026] poll method: The poll method usually appears in the implementation of blocking queues (such as the BlockingQueue interface in Java) and is used to remove and return the element at the head of the queue. Different from the take method, the poll method is non-blocking, that is, if the queue is empty, the poll method will not block the calling thread waiting for data to arrive, but immediately return null or a special value in the queue (such as -1) to indicate that the queue is currently empty. The usage scenarios are mainly those occasions that need to regularly check the queue status but do not want to wait indefinitely.

[0027] Embodiment 1

[0028] According to an embodiment of the present application, a method for managing service information is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from here.

[0029] Figure 1 is a schematic flowchart of a method for managing service information provided according to an embodiment of the present application, as Figure 1 shown, the method includes the following steps:

[0030] Step S102, using a pre-configured aspect collection plug-in to collect service system call information, collect record information of application program interfaces, and write the record information into a memory queue, where the type of the memory queue is a consumption blocking queue.

[0031] Common aspect collection plugins include, but are not limited to, the following: Spring AOP aspect plugin, AspectJ aspect plugin, Log4j Interceptor, etc. The main purpose of the aspect collection plugin is to insert specific logic into the business code without directly modifying the business code, thus maintaining the clarity and modularity of the business code.

[0032] These plugins can execute the same logic (aspect) at multiple positions (join points) in the code, enabling developers to focus on writing business logic and leaving common functions such as logging and performance monitoring to the aspect plugins. This not only simplifies the code but also improves maintainability and scalability.

[0033] A memory queue is a data structure implemented in program memory, mainly used for handling data flow control in concurrent scenarios. It is usually designed to be thread-safe and can support data production and consumption in a multi-threaded environment. Memory queues are very useful in various scenarios. For example, in the producer-consumer model, the producer thread is responsible for generating data, and the consumer thread is responsible for processing data. The two exchange data through a memory queue, thus achieving asynchronous processing and decoupling of data. The specific implementation of the memory queue can be based on data structures such as arrays, linked lists, or circular buffers. In high-concurrency scenarios, to ensure data consistency and avoid race conditions, memory queues often need to adopt certain synchronization mechanisms, such as locks, semaphores, atomic variables, etc., to ensure thread safety. Common memory queues are synchronous blocking queues, which will automatically block threads when the queue is empty or full until there is data to consume or space to write in the queue.

[0034] In step S104, use a pre-configured first thread to read the record information in the memory queue and write the read record information into the list cache corresponding to the first thread. When the list cache reaches a preset condition, write all the record information in the list cache into the log file on the local disk, where the preset condition includes at least one of the following: data volume threshold, waiting time threshold.

[0035] In actual use, the data volume threshold refers to the minimum data volume that the system needs to reach before performing a specific operation. Setting the data volume threshold can effectively reduce disk I / O operations and optimize the system memory. In the above process, the data volume threshold is used to control writing the data in the memory queue to the local file when the data volume in the list cache reaches the preset value; the waiting time threshold refers to the maximum time that the system needs to wait before performing a specific operation. In the above process, the waiting time threshold is used to control when to write the data in the memory queue to the local file.

[0036] When setting the data volume threshold and the waiting time threshold, the utilization rate of system resources and the real-time nature of data processing need to be considered. For example, if the data volume threshold is set relatively high, the waiting time threshold can be set relatively low to ensure that even if the data volume does not reach the threshold, the system will perform data writing operations regularly to avoid data backlog. On the contrary, if the data volume threshold is low, the waiting time threshold can be set high to reduce unnecessary file and database writing operations and save system resources.

[0037] Among them, the first thread mainly realizes the work of writing the recorded information from the memory queue to the list cache and then from the list cache to the local file. The pre-configured first thread transfers the recorded information to the log file on the local disk through the list cache, rather than directly writing to the database, allowing the business processing and data persistence to be independent of each other, which provides the possibility for subsequent asynchronous decoupling.

[0038] Step S106, use the pre-configured second thread to periodically read the incremental log files on the local disk and batch-write all the incremental log files into the target database.

[0039] Among them, the second thread reads the newly generated log files since the last read at a preset time interval. The incremental log file refers to the log data generated within a specific time interval and only contains the newly added data since the last processing. Using incremental log files can reduce the amount of data to be processed, and periodic reading can ensure the timely processing of data and improve the real-time nature of data processing.

[0040] The pre-configured second thread refers to an independent thread created and configured in advance during the system initialization phase, and its main responsibility is to periodically perform log file reading and database writing operations. This thread runs independently of the business thread, avoiding the possible performance impact of directly performing these operations in the business thread.

[0041] In the embodiment of the present application, use the pre-configured aspect collection plug-in to collect the recorded information in the business system in a non-invasive manner. In an environment with limited resources, adopt a dual-thread mechanism, use the local disk file as the data buffer, adopt a multi-level storage strategy from the memory queue to the local file and then to the database, and finally write the incremental log files into the target database, thus realizing the efficient and accurate collection and storage of key business information in a high-concurrency scenario.

[0042] The following describes each step of the business information management method in combination with a specific implementation process.

[0043] Before data collection and acquisition, first, relevant software, hardware, and parameters need to be configured to build a relatively complete information collection environment.

[0044] As an alternative implementation, before collecting the record information of the business system call information collection application interface using the pre-configured aspect collection plug-in, a series of key configuration steps required include the following aspects:

[0045] Step S1, configure a general application interface and a deduplication interface for collecting business information that interacts with the business system, and form an information collection application interface based on the general application interface and the deduplication interface.

[0046] Before collecting business information, it is first necessary to define a set of general application interfaces for interacting with the business system and collecting call information. These application interfaces need to be designed flexibly enough to adapt to the needs of different business systems, while ensuring data consistency and integrity. For example, the general API can include collecting user information, business operation logs, exception information, etc. When designing, the unity of data format should be considered, such as using JSON or XML as the data exchange format for subsequent processing and storage.

[0047] Since there may be duplicate call information in the business system, such as multiple attempts at the same operation or redundant messages within the system, it is necessary to configure a deduplication interface to filter out duplicate data. The deduplication logic can be implemented based on unique identifiers such as timestamps, operation IDs, or user IDs to ensure that the stored data set does not contain redundant information, improving storage efficiency and the accuracy of data analysis.

[0048] Step S2, configure the aspect points and initialization methods corresponding to the aspect collection plug-in.

[0049] Among them, according to business requirements, configure the aspect points of the aspect collection plug-in, that is, at which positions in the business system code to insert the aspect logic. The aspect points should cover all key business call paths to ensure that important business information can be accurately captured.

[0050] Optionally, to ensure that there is only one instance of the aspect collection plug-in in the system (i.e., implement the singleton pattern) and avoid resource competition and data inconsistency problems between multiple instances, the initialization method of the plug-in can be configured to start as a singleton.

[0051] Step S3, configure the first configuration parameters corresponding to the first thread.

[0052] Optionally, the first configuration parameter includes parameters such as the queue consumption method and preset conditions. Among them, the queue consumption method is the strategy for reading data from the memory queue. For example, the poll method can be used to read data immediately, or the take method can be used to block and wait when the queue is empty. Considering the high-concurrency business processing scenario, preferentially choosing the poll method as the strategy for reading data helps to keep the business threads active, ensuring the smoothness of the business processing flow and the controllability of the response time. The preset conditions reflect the conditions for setting off data writing to the local file, including data volume thresholds and waiting time thresholds, etc., ensuring that data can be written to the file in a timely and batch manner.

[0053] Step S4, configure the second configuration parameter corresponding to the second thread.

[0054] Optionally, the second configuration parameter includes batch writing parameters. Among them, the batch writing parameters reflect the strategy for batch writing data to the database. For example, the obtained log files can be grouped according to a preset number of records, and each group is used as a batch to be written to the target data in batches. This helps to optimize the database writing efficiency, reduce network latency and database load.

[0055] The above steps S1 - S4 of the configuration jointly ensure the efficiency and stability of the entire data collection and storage process. Through the carefully designed general interface and deduplication interface, it can be ensured that the collected information is both comprehensive and redundant-free; the singleton initialization method and the setting of the aspect point ensure the correct deployment of the plugin and the accurate capture of business call information; the configuration parameters of the first thread and the second thread respectively control the transfer of data from the memory to the file and from the file to the database, realizing the asynchronous processing and efficient writing of data, and ultimately achieving the purpose of improving the business information collection and processing capabilities under resource constraints. This configuration method is not only applicable to the current business scenario, but also has good scalability and adaptability, and can be flexibly adjusted as the business requirements change.

[0056] After the relevant pre-configuration work is completed, the following method can be used to collect the record information of the business system call information collection application interface by using the pre-configured aspect collection plugin and write the record information into the memory queue: Singleton start the aspect collection plugin based on the double-checked locking mechanism, initialize the memory queue, use the aspect collection plugin to collect the record information of the business system call information collection application interface, and write the record information into the initialized memory queue.

[0057] Among them, the double-checked locking mechanism, as an optimization method to ensure thread safety in the singleton pattern in a multi-threaded environment, can avoid the problem of initializing an object multiple times in a concurrent environment when starting the aspect collection plug-in. Through this mechanism, it can be ensured that there is only one instance of the aspect collection plug-in in the system, avoiding resource waste and potential synchronization problems. At the same time, since the use of locks can be avoided in most cases, the startup efficiency of the plug-in is improved.

[0058] The memory queue, as a key component for temporarily storing business system call information, initializing the memory queue includes setting the size, type (such as a blocking queue), and management strategy of the queue. When actually performing the initialization operation, the capacity and consumption strategy of the queue should be considered to ensure that the queue can work stably in a high-concurrency environment.

[0059] In the above process, the collected record information can include the timestamp of the business call, call parameters, return results, exception information, etc. These information are crucial for subsequent analysis and problem location. Writing the record information into the memory queue can be achieved in the following way: encapsulating the record information based on a preset standard data format to obtain standard business information, and writing the standard business information into the memory queue.

[0060] Optionally, the record information includes business processing flow information and business exception information collected from the business system. After collecting this information, the record information is encapsulated using a preset standard data format and converted into a unified data structure. The encapsulation process can include the following steps: data parsing, data verification, standardization encapsulation, exception handling, etc. Encapsulating the record information using a preset standard data format can ensure that the collected business information is not only comprehensive but also consistent, facilitating subsequent data processing and analysis.

[0061] After writing the record information into the memory queue, the record information needs to be read and cached in the corresponding list. Specifically, it can be achieved in the following way: using the first thread to read the record information from the memory queue through the poll method, performing duplicate removal processing on the read record information, and writing the de-duplicated record information into the list cache corresponding to the first thread.

[0062] In the above process, the poll method is used for the reading process of the record information. Since the poll method is non-blocking, it ensures that the first thread can regularly check the queue without wasting CPU time waiting for the queue to become full. In addition, the use of the poll method allows the first thread to return immediately when the queue is empty, process other tasks or enter a waiting state, thus improving the overall responsiveness and flexibility of the system.

[0063] Considering the repeated operations of users or the retry mechanism of the system, duplicate record information may be generated. The read record information needs to be further deduplicated. The following methods can be used to achieve deduplication in practice: Generate a unique identifier for each record (such as a hash value based on timestamp, business ID, and user ID), compare the generated unique identifier with the existing records in the list cache. If a record with the same identifier is found, it can be determined as duplicate information and filtered out from the current read results. The deduplicated record information will be written into the list cache corresponding to the first thread as preparation for subsequent batch writing to the database.

[0064] To ensure data persistence and reliability, when the above list cache reaches relevant preset conditions, all record information in the list cache will be written into a log file on the local disk. Optionally, multiple batches of record information can be written into one log file, or each batch of record information can be written into a separate log file.

[0065] Among them, the preset conditions can be a data volume threshold and a waiting time threshold. These two factors are crucial for controlling memory usage and ensuring timely data persistence. Too small a data volume threshold will lead to frequent disk writes, increasing I / O overhead; while too large a data volume threshold may occupy too much memory and affect system performance. In addition, setting the waiting time threshold helps prevent data from staying in memory for too long, ensuring data timeliness and system stability.

[0066] For the generated log files, a pre-configured second thread can be used to periodically read the incremental log files on the local disk and batch write all incremental log files into the target database. Specifically, it can be achieved through the following methods: Use the second thread to periodically compare the log files on the local disk and in the target database to determine the incremental log files on the local disk, group the incremental log files according to the pre-configured batch writing parameters, and batch write the grouped incremental log files into the target database based on the remote procedure call interface.

[0067] Optionally, based on the situation where all record information is written into one log file from beginning to end, the content of the log file on the local disk can be directly compared with the existing record information in the database. If the content of a certain log file on the local disk does not appear in the target database, it means that this log file is a new incremental log file. This method of directly comparing log content to determine incremental log files can effectively ensure data integrity, but it will consume more computing resources and I / O operations in the case of a large log file content or a large number of database record information.

[0068] Optionally, based on the above problems, it can be considered to create and generate multiple log files on the local disk, which can be specifically implemented in the following way: when the data volume in the list cache reaches the data volume threshold or the waiting time corresponding to the list cache reaches the waiting time threshold, a log file is created in the local disk, all record information in the list cache is written into the log file, and the list cache is cleared. When the next trigger condition is met, the above steps are repeated. In this process, as long as the relevant preset conditions are reached, a new log file is created. As time goes by or the data volume accumulates, multiple log files will be generated. Assuming that each log file is named according to the creation timestamp, therefore, the naming rule reflects the corresponding creation order. The larger the timestamp, the newer the log file.

[0069] In the above process, there are serial numbers with timestamp naming corresponding to the multiple generated log files. When determining the incremental log files, the serial numbers of the log files in the target database and the local disk can be compared, which can be specifically implemented in the following way: maintain a log file storage record in the target database to store and process the serial number of the latest log file stored in the database. In actual application, the log files generated on the local disk can be named with a serial number with a timestamp. Then, when actually comparing, first check the serial number of the latest log file in the storage record in the current target database. Then, the log files on the local disk with serial numbers after the above serial number are the incremental log files.

[0070] After determining the incremental files, group the incremental log files according to the batch write parameters. For each group of data, perform a batch write operation based on the remote procedure call interface and insert the data into the corresponding table in the database. Batch write operations are usually more efficient than single record writes because they reduce the number of interactions with the database.

[0071] The aspect collection plug-in and the dual-thread mechanism proposed in the embodiments of the present application achieve efficient collection, storage, and batch writing of business information, effectively reducing the consumption of system resources, improving the efficiency and reliability of data processing. Secondly, the double-checked locking mechanism ensures the singleton startup of the aspect collection plug-in, avoiding resource contention. The design of the consumption blocking queue enables the system to automatically adjust the consumption speed when the data volume is large, avoiding data loss. In addition, through the setting of preset conditions, the writing timing of data can be flexibly controlled, further optimizing the storage and writing efficiency. Thus, it effectively solves the technical problem of difficultly obtaining relevant business information efficiently when verifying user services in the complaint management scenario.

[0072] Embodiment 2

[0073] According to the embodiments of the present application, there is also provided a business information management device for implementing the business information management method in Embodiment 1, asFigure 2 As shown in Figure 2 , the management device for the service information at least includes: a collection module 21, a first storage module 22, and a second storage module 23, where:

[0074] The collection module 21 can collect the record information of the service system call information collection application interface by using a pre-configured aspect collection plug-in, and write the record information into a memory queue, where the type of the memory queue is a consumption blocking queue.

[0075] The first storage module 22 can use a pre-configured first thread to read the record information in the memory queue, and write the read record information into the list cache corresponding to the first thread. When the list cache reaches a preset condition, all the record information in the list cache is written into the log file on the local disk, where the preset condition includes at least one of the following: a data volume threshold, a waiting time threshold.

[0076] The second storage module 23 can use a pre-configured second thread to periodically read the incremental log file on the local disk, and batch write all the incremental log files into the target database.

[0077] The functions of each module of the management device for the service information are described below in combination with a specific implementation process.

[0078] Optionally, before collecting information, the collection module needs to configure relevant software and hardware parameters. The specific configuration operations include the following steps: configuring a general application interface and a deduplication interface for collecting service information for interacting with the service system, and forming an information collection application interface based on the general application interface and the deduplication interface; configuring the aspect point and initialization method corresponding to the aspect collection plug-in; configuring the first configuration parameters corresponding to the first thread; configuring the second configuration parameters corresponding to the second thread.

[0079] Among them, the initialization method is singleton startup. The first configuration parameters include but are not limited to: queue consumption method, preset condition. The second configuration parameters include but are not limited to batch write parameters.

[0080] After the collection module obtains the record information, the record information can be written into the memory queue in the following manner: starting the aspect collection plug-in as a singleton based on the double-checked locking mechanism; initializing the memory queue; using the aspect collection plug-in to collect the record information of the service system call information collection application interface, and writing the record information into the initialized memory queue.

[0081] Among them, writing the record information into the memory queue can be implemented in the following manner: encapsulating the record information based on a preset standard data format to obtain standard service information, where the record information includes service processing flow information and service exception information; writing the standard service information into the memory queue.

[0082] After the collection module collects the record information and writes it into the memory queue, the first storage module can use a pre-configured first thread to read the record information in the memory queue and write the read record information into the list cache corresponding to the first thread.

[0083] As an optional implementation, writing the read record information into the list cache can be achieved through the following methods: use the first thread to read the record information in the memory queue through the poll method; perform deduplication processing on the read record information; write the deduplicated record information into the list cache corresponding to the first thread.

[0084] To ensure data persistence and reliability, when the above list cache reaches relevant preset conditions, all the record information in the list cache is written into a log file on the local disk, where the log file can store the record information in multiple batches of list caches.

[0085] As an optional implementation, multiple log files can also be created and generated in batches on the local disk, which can be specifically achieved through the following methods: when the data volume in the list cache reaches the data volume threshold or the waiting time corresponding to the list cache reaches the waiting time threshold, create a log file on the local disk, write all the record information in the list cache into the log file, and clear the list cache. When the next trigger condition is met, continue to repeat the above steps.

[0086] After the first storage module generates the log file, the second storage module uses a pre-configured second thread to periodically read the incremental log files on the local disk and batch-writes all the incremental log files into the target database.

[0087] Among them, when determining the incremental log file, it can be done in the following ways:

[0088] Optionally, for the case where all the record information is written into a single log file from beginning to end, the content of the log file on the local disk can be directly compared with the existing record information in the database. If the content of a certain log file on the local disk does not appear in the target database, it means that the log file is a new log file.

[0089] Optionally, for the case where multiple log files are created and generated on the local disk, the log file numbers of the target database and the local disk can be compared. A log file entry record is maintained in the target database to store and process the number of the latest log file for warehousing. In actual applications, the log files generated on the local disk can be named with a sequence number with a timestamp. Then, during actual comparison, first check the number of the latest log file in the entry record in the current target database. Then, the log files on the local disk with sequence numbers after the above sequence number are the incremental log files.

[0090] As an alternative implementation, the second storage module can batch write the incremental log file into the target database in the following way: use the second thread to periodically compare the log files in the local disk and the target database to determine the incremental log files in the local disk; group the incremental log files according to the pre-configured batch write parameters; and batch write the grouped incremental log files into the target database based on the remote procedure call interface.

[0091] It should be noted that each module in the business information management device in the embodiments of the present application corresponds one by one to each implementation step of the business information management method in Embodiment 1. Since Embodiment 1 has been described in detail, some details not shown in this embodiment can be referred to Embodiment 1 and will not be elaborated here.

[0092] Embodiment 3

[0093] According to the embodiments of the present application, there is also provided a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the business information management method in Embodiment 1.

[0094] According to the embodiments of the present application, there is also provided a non-volatile storage medium, which includes a stored computer program. The device where the non-volatile storage medium is located executes the business information management method in Embodiment 1 by running the computer program.

[0095] According to the embodiments of the present application, there is also provided a processor, which is used to run a computer program. When the computer program runs, it executes the business information management method in Embodiment 1.

[0096] According to the embodiments of the present application, there is also provided an electronic device, which includes: a memory and a processor. The memory stores a computer program, and the processor is configured to execute the business information management method in Embodiment 1 through the computer program.

[0097] Specifically, when the computer program runs, it implements the following steps: collecting business system call information and record information of application program interfaces by using a pre-configured aspect collection plugin, and writing the record information into a memory queue, where the type of the memory queue is a consumption blocking queue; reading the record information in the memory queue by using a pre-configured first thread, and writing the read record information into a list cache corresponding to the first thread. When the list cache meets a preset condition, writing all the record information in the list cache into a log file on the local disk, where the preset condition includes at least one of the following: a data volume threshold, a waiting time threshold; periodically reading incremental log files on the local disk by using a pre-configured second thread, and batch writing all the incremental log files into a target database.

[0098] As an optional implementation manner, the above electronic device may exist in the form of a mobile terminal, a computer terminal, or a similar computing device. Figure 3 The hardware structure block diagram of an electronic device for implementing a method for managing service information is shown. As Figure 3 shown, the electronic device 30 may include one or more (shown as 302a, 302b,..., 302n in the figure) processors 302 (the processor 302 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 304 for storing data, and a transmission device 306 for communication functions. In addition, it may further include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply, and / or a camera. Those of ordinary skill in the art can understand that Figure 3 the structure shown is only schematic and does not limit the structure of the above electronic device. For example, the electronic device 30 may further include more or fewer components than those Figure 3 shown, or have a different configuration from that Figure 3 shown.

[0099] It should be noted that the above one or more processors 302 and / or other data processing circuits are generally referred to as "data processing circuits" in this article. The data processing circuit may be fully or partially embodied as software, hardware, firmware, or any arbitrary combination thereof. In addition, the data processing circuit may be a single independent processing module, or fully or partially incorporated into any one of the other elements in the electronic device 30. As involved in the embodiments of the present application, the data processing circuit is a processor control (such as the selection of a variable resistance terminal path connected to an interface).

[0100] The memory 304 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the business information management method in the embodiments of the present application. The processor 302 executes various functional applications and data processing by running the software programs and modules stored in the memory 304, that is, implements the vulnerability detection method of the above application program. The memory 304 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 304 may further include a memory remotely disposed relative to the processor 302, and these remote memories can be connected to the electronic device 30 through a network. Examples of the above network include but are not limited to the Internet, enterprise intranet, local area network, mobile communication network, and combinations thereof.

[0101] The transmission device 306 is used to receive or send data via a network. Specific examples of the above network may include the wireless network provided by the communication provider of the electronic device 30. In one instance, the transmission device 306 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station and thus can communicate with the Internet. In one instance, the transmission device 306 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0102] The display can be, for example, a touch-screen liquid crystal display (LCD), and the liquid crystal display enables the user to interact with the user interface of the electronic device 30.

[0103] The above serial numbers of the embodiments are only for description and do not represent the superiority or inferiority of the embodiments.

[0104] In the above embodiments of the present application, the descriptions of each embodiment have their own focuses. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0105] In several embodiments provided in the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only illustrative. For example, the division of units can be a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces, and the indirect coupling or communication connection of units or modules can be in an electrical or other form.

[0106] The unit described as a separating component may or may not be physically separated, and the component shown as a unit may or may not be a physical unit, that is, it may be located in one place or distributed to multiple units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0107] In addition, each functional unit in various embodiments of the present application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0108] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that makes a contribution to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the methods in various embodiments of the present application. The foregoing storage medium includes: USB flash drives, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disks, magnetic disks or optical discs and other various media that can store program codes.

[0109] The above are only the preferred embodiments of the present application. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. A method for managing service information, characterized in that, Including: Collecting the record information of the application programming interface by using a pre-configured aspect collection plug-in to collect the business system call information, and writing the record information into a memory queue, where the type of the memory queue is a consumption blocking queue; Using a pre-configured first thread to read the record information in the memory queue, and writing the read record information into the list cache corresponding to the first thread. When the list cache reaches a preset condition, writing all the record information in the list cache into a log file on the local disk, where the preset condition includes at least one of the following: data volume threshold, waiting time threshold; Using a pre-configured second thread to periodically read the incremental log files on the local disk, and batch writing all the incremental log files into a target database.

2. The method according to claim 1, wherein Before collecting the record information of the application programming interface by using a pre-configured aspect collection plug-in to collect the business system call information, the method further includes: Configuring a general application programming interface for collecting business information and a deduplication interface for interacting with the business system, and forming the information collection application programming interface based on the general application programming interface and the deduplication interface; Configuring the aspect point and initialization method corresponding to the aspect collection plug-in, where the initialization method is singleton startup; Configuring the first configuration parameters corresponding to the first thread, where at least the first configuration parameters include: queue consumption method, the preset condition; Configuring the second configuration parameters corresponding to the second thread, where at least the second configuration parameters include: batch writing parameters.

3. The method according to claim 1, wherein Collecting the record information of the application programming interface by using a pre-configured aspect collection plug-in to collect the business system call information, and writing the record information into a memory queue, including: Starting the aspect collection plug-in as a singleton based on the double-checked locking mechanism; Initializing the memory queue; Using the aspect collection plug-in to collect the record information of the business system calling the information collection application programming interface, and writing the record information into the initialized memory queue.

4. The method according to claim 1 or 3, characterized in that, Writing the record information into the memory queue, including: Encapsulating the record information based on a preset standard data format to obtain standard business information, where the record information includes business processing flow information and business exception information; Writing the standard business information into the memory queue.

5. The method according to claim 1, wherein Using a pre-configured first thread to read the record information in the memory queue, and writing the read record information into the list cache corresponding to the first thread, including: Using the first thread to read the record information in the memory queue through the poll method; Performing deduplication processing on the read record information; Writing the deduplicated record information into the list cache corresponding to the first thread.

6. The method according to claim 1, wherein When the list cache reaches a preset condition, writing all the record information in the list cache into a log file on the local disk, including: When the amount of data in the list cache reaches the data volume threshold or the waiting time corresponding to the list cache reaches the waiting time threshold, a log file is created in the local disk, all record information in the list cache is written into the log file, and the list cache is cleared.

7. The method according to claim 1 or 6, characterized in that Periodically read the incremental log files in the local disk by using a pre-configured second thread, and batch-write all the incremental log files into the target database, including: Periodically compare the log files in the local disk and the target database by using the second thread to determine the incremental log files in the local disk; Group the incremental log files according to pre-configured batch write parameters; Batch-write the grouped incremental log files into the target database based on the remote procedure call interface.

8. A management device for service information, characterized in that, Including: A collection module, configured to collect the record information of the application program interface by using a pre-configured aspect collection plug-in to collect the business system call information, and write the record information into a memory queue, where the type of the memory queue is a consumption blocking queue; A first storage module, configured to read the record information in the memory queue by using a pre-configured first thread, and write the read record information into the list cache corresponding to the first thread. When the list cache meets a preset condition, all the record information in the list cache is written into the log file in the local disk, where the preset condition includes at least one of the following: data volume threshold, waiting time threshold; A second storage module, configured to periodically read the incremental log files in the local disk by using a pre-configured second thread, and batch-write all the incremental log files into the target database.

9. A computer program product, characterized in that, Including: A computer program, where when the computer program is executed by a processor, it implements the management method of business information described in any one of claims 1 to 7.

10. An electronic device, characterized in that, Including: A memory and a processor, where a computer program is stored in the memory, and the processor is configured to execute the management method of business information described in any one of claims 1 to 7 through the computer program.

Images