Slow log collection method and device, equipment and storage medium

Abstract

The embodiment of the invention discloses a slow log collection method and device, equipment and a storage medium, and relates to the technical field of databases, the method comprises the following steps: determining a current analysis state of each process in a distributed document storage database, and starting an analysis function of the process of which the current analysis state is started; collecting a slow log of each process of which the analysis function is started, and storing the slow log of each process based on a log file; and determining a target slow log in the log file according to the received query request. According to the technical scheme, the analysis function of the process of which the current analysis state is started is started, the slow log of the process of which the analysis function is started is collected when the distributed document storage database runs, the collected slow log is stored in the log file, after the query request is received, the target slow log is determined in the log file according to the query request, and the target slow log is sent to the distributed document storage database. And on the premise of reducing the influence on the performance of the distributed document storage database, the collection and query of the slow log are realized.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of database technology, and in particular to a slow log collection method, device, equipment and storage medium. Background Art

[0002] In the context of cloud computing and big data, database applications in public cloud environments are becoming more and more widespread. As the database runs, the amount of database business access increases, and the amount of database data increases, query results often return slowly, so it is necessary to record slow logs to find the slow operations that form bottlenecks. The distributed document storage database (MongoDB) is between relational databases and non-relational databases and can store more complex data types.

[0003] In the prior art, MongoDB starts the analyzers of all MongoD instances through startup parameters, and collects MongoDB slow logs through the analyzers of all MongoD instances.

[0004] In the process of implementing the present invention, the inventors found that there are at least the following technical problems in the prior art:

[0005] Keeping the profiler enabled for all MongoD instances for a long time can affect MongoDB performance. Summary of the invention

[0006] The present invention provides a slow log collection method, device, equipment and storage medium, which realize the collection of slow logs under the premise of reducing the impact on MongoDB performance.

[0007] In a first aspect, an embodiment of the present invention provides a slow log collection method, including:

[0008] Determine the current analysis status of each process in the distributed document storage database, and enable the analysis function of the process whose current analysis status is enabled;

[0009] Collect the slow log of each process with the analysis function enabled, and store the slow log of each process based on the log file;

[0010] A target slow log is determined in the log file according to the received query request.

[0011] In a second aspect, an embodiment of the present invention further provides a slow log collection device, including:

[0012] An enabling module, used to determine the current analysis status of each process in the distributed document storage database, and enable the analysis function of the process whose current analysis status is enabled;

[0013] A collection module, used for collecting the slow logs of each process with the analysis function enabled, and storing the slow logs of each process based on log files;

[0014] The determination module is used to determine the target slow log in the log file according to the received query request.

[0015] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:

[0016] one or more processors;

[0017] a storage device for storing one or more programs,

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the slow log collection method as described in any one of the first aspects.

[0019] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer executable instructions, wherein the computer executable instructions are used to execute the slow log collection method as described in any one of the first aspects when executed by a computer processor.

[0020] The embodiments of the above invention have the following advantages or beneficial effects:

[0021] The embodiment of the present invention provides a slow log collection method, including: determining the current analysis status of each process in a distributed document storage database, and opening the analysis function of the process whose current analysis status is turned on; collecting the slow logs of each process whose analysis function is turned on, and storing the slow logs of each process based on the log file; and determining the target slow log in the log file according to the received query request. In the above technical scheme, after determining the current analysis status of each process in MongoDB, the control system of MongoDB turns on the analysis function of the process whose current analysis status is turned on, so that when MongoDB is running, it can collect the slow log of the process based on the analysis function of the process, collect the slow log of each process whose analysis function is turned on, and store the slow log of each process based on the log file to realize the collection of slow logs. After receiving the query request, the target slow log is determined in the log file according to the query request to realize the query of the slow log, without turning on the analysis function of the process in MongoDB, to realize the collection and query of the slow log, and reduce the impact on the performance of MongoDB. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A flowchart of a slow log collection method provided by an embodiment of the present invention;

[0023] Figure 2A flowchart of another slow log collection method provided by an embodiment of the present invention;

[0024] Figure 3 A schematic diagram of the structure of a slow log collection device provided by an embodiment of the present invention;

[0025] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0026] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0027] It should be mentioned before discussing exemplary embodiments in more detail that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe various operations (or steps) as sequential processes, many operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of various operations can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to methods, functions, procedures, subroutines, subprograms, etc. In addition, the embodiments in the present invention and the features in the embodiments can be combined with each other without conflict.

[0028] To collect slow logs for MongoDB, you can enable the analyzer of all MongoD instances through startup parameters or configuration files to collect slow logs for all MongoD instances in MongoDB. You can also enable the analyzer of the MongoD instance that needs to collect slow logs through instructions to collect its slow logs.

[0029] Although the analyzer can record the operation details of the MongoD instance, it has an impact on the performance of MongoDB. For MongoDB with high load, it is necessary to avoid opening the analyzer for a long time as much as possible.

[0030] The first method of collecting slow logs has a certain impact on MongoDB performance. The second method of collecting slow logs requires that you use instructions to enable the analyzer of the MongoD instance that needs to collect slow logs, and you cannot enable the analyzer of the MongoD instance in the newly created database.

[0031] Therefore, this application proposes a slow log collection method to achieve slow log collection without opening the analyzers of all MongoD instances, which not only achieves slow log collection but also reduces the impact on MongoDB performance.

[0032] The slow log collection method proposed in this application will be described in detail below with reference to diagrams and embodiments.

[0033] Figure 1 The flowchart of a slow log collection method provided by an embodiment of the present invention is applicable to the situation where slow log collection needs to be implemented under the premise of reducing the impact on MongoDB performance. The method can be executed by a slow log collection device, which can be implemented by software and / or hardware. Figure 1 Said method specifically comprises the following steps:

[0034] Step 110: determine the current analysis status of each process in the distributed document storage database, and enable the analysis function of the process whose current analysis status is enabled.

[0035] MongoDB is an open source, high-performance, schema-free distributed document storage database that supports BSON data structures similar to JSON and can store relatively complex data types. The analysis status of a process in MongoDB can be on or off, and its analysis status can be determined by the user triggering the analysis control corresponding to the process.

[0036] Specifically, the user terminal displays each process in MongoDB based on a preset interface, and an analysis control corresponding to each process is set at the corresponding position of the preset interface for displaying each process. When the last analysis state of the analysis control corresponding to the process is on, the user triggers the analysis control to determine that the current analysis state of the process is off; when the last analysis state of the analysis control corresponding to the process is off, the user triggers the analysis control to determine that the current analysis state of the process is on. Therefore, the current analysis state of each process can be determined based on the last analysis state of each process and whether a trigger operation for each process is received.

[0037] After determining the analysis status of each process according to the triggering of the analysis control corresponding to each process by the user, the analysis status of each process can be stored in the metadata database based on the profile_status table. Furthermore, the control system of MongoDB can determine the analysis status of each process by accessing the profile_status table in the metadata database.

[0038] After determining the current analysis status of each process, if the current analysis status of the process is enabled, it indicates that slow log collection is required for the process. For the control system of MongoDB, the slow log of the process can be collected by enabling the analysis function of the process. Therefore, after determining that the current analysis status of the process is enabled, the analysis function of the process can be enabled.

[0039] In an embodiment of the present invention, by determining the current analysis status of each process in MongoDB, it is determined whether it is necessary to collect the slow log of each process, and the analysis function of the process for which the slow log needs to be collected is enabled, so that when MongoDB is running, the slow log of the process can be collected based on the process analysis function.

[0040] Step 120: Collect the slow logs of each process with the analysis function enabled, and store the slow logs of each process based on the log file.

[0041] Among them, the slow log can be understood as the slow operation log whose execution time exceeds the preset threshold, which is filtered out from all operations of the process after the analysis function is enabled.

[0042] Specifically, the analysis function of the process that needs to collect slow logs has been turned on. Therefore, when MongoDB is running, the control system of MongoDB can collect the slow logs generated during the running of the process that needs to collect slow logs based on the analysis function of the process that needs to collect slow logs, and store the collected slow logs in the corresponding system.profile collection, thereby realizing the slow log collection of the process that needs to collect slow logs.

[0043] Furthermore, the collected slow logs can be stored in log files. The collected slow logs are BSON format documents consisting of multiple fields. Therefore, each collected slow log can be converted into a string in JSON format and stored in a fixed log file, which can be stored on the disk of the MongoDB control system.

[0044] In an embodiment of the present invention, the control system of MongoDB collects slow logs generated during the running of the process that needs to collect slow logs based on the analysis function of the process that needs to collect slow logs, thereby realizing the collection of slow logs of the process that needs to collect slow logs, and realizing the storage of the collected slow logs based on the log files.

[0045] Step 130: Determine a target slow log in the log file according to the received query request.

[0046] The query request may be triggered by a user based on a user terminal, and the query request may include a user identifier. It is understandable that after receiving the query request, it is necessary to determine the slow log of the process created by the user corresponding to the user identifier included in the query request.

[0047] Specifically, the user identifier contained in the received query request can be determined first, and then the target log file can be determined in the log file according to the user identifier contained in the query request. Furthermore, the target slow log can be determined in the target log file. In the process of determining the target slow log, the current slow log can be determined in the target log file according to the current query timestamp and the query capacity that does not affect the performance of MongoDB. If the current slow log contains the latest slow log, the current slow log is determined as the target slow log. If the current slow log does not contain the latest slow log, the next query timestamp is determined according to the current query timestamp and the query capacity, and the next query timestamp is used as the current query timestamp. The current slow log is continued to be determined in the target log file according to the current query timestamp and the query capacity until the current slow log contains the latest slow log. At this time, all the current slow logs determined in the target log file can be determined as the target slow log, so that the slow log of the process created by the user corresponding to the user identifier contained in the query request is determined as the target slow log.

[0048] In the embodiment of the present invention, it is achieved to determine the target slow log corresponding to the query request in the log file according to the received query request.

[0049] The slow log collection method provided by the embodiment of the present invention includes: determining the current analysis status of each process in the distributed document storage database, and opening the analysis function of the process whose current analysis status is turned on; collecting the slow logs of each process whose analysis function is turned on, and storing the slow logs of each process based on the log file; and determining the target slow log in the log file according to the received query request. In the above technical scheme, after determining the current analysis status of each process in MongoDB, the control system of MongoDB turns on the analysis function of the process whose current analysis status is turned on, so that when MongoDB is running, it can collect the slow log of the process based on the analysis function of the process, collect the slow log of each process whose analysis function is turned on, and store the slow log of each process based on the log file to realize the collection of slow logs. After receiving the query request, the target slow log is determined in the log file according to the query request to realize the query of the slow log, without turning on the analysis function of the process in MongoDB, to realize the collection and query of the slow log, and reduce the impact on the performance of MongoDB.

[0050] Figure 2 This is a flowchart of another slow log collection method provided by an embodiment of the present invention. The embodiment of the present invention is applicable to the situation where slow log collection needs to be implemented under the premise of reducing the impact on MongoDB performance. The embodiment of the present invention is concretized on the basis of the above embodiment, and the explanations of the terms that are the same or corresponding to the above embodiments are not repeated here. Figure 2 , the slow log collection method provided by the embodiment of the present invention includes:

[0051] Step 210: Determine the current analysis status of each process in the distributed document storage database.

[0052] In one implementation, step 210 may specifically include:

[0053] The current analysis status of each process is determined according to the received trigger operation for each process and the previous analysis status of each process.

[0054] The process in MongoDB can be understood as a MongoD instance. A MongoD instance is an independent running unit on MongoDB, responsible for processing data requests related to the instance and managing data storage. By configuring and starting multiple MongoD instances, you can achieve MongoDB's high availability, load balancing, and data sharding functions. MongoD instances can be replica set type instances or sharded cluster type instances.

[0055] Specifically, the user terminal displays each MongoD instance in MongoDB based on a preset interface, and the analysis control corresponding to each MongoD instance is set at the corresponding position of the preset interface for displaying each MongoD instance. When the last analysis state of the analysis control corresponding to the MongoD instance is on, the user's trigger operation on the analysis control corresponding to the MongoD instance can determine that the current analysis state of the MongoD instance is off; when the last analysis state of the analysis control corresponding to the MongoD instance is off, the user's trigger operation on the analysis control corresponding to the MongoD instance can determine that the current analysis state of the MongoD instance is on. Therefore, the current analysis state of each MongoD instance can be determined based on the last analysis state of each MongoD instance and whether the user's trigger operation for each MongoD instance is received.

[0056] Of course, after determining the current analysis status of each MongoD instance according to the triggering of the analysis control corresponding to each MongoD instance by the user, the current analysis status of each MongoD instance can be stored in the metadata database based on the profile_status table. Furthermore, the control system of MongoDB can determine the current analysis status of each MongoD instance by accessing the profile_status table in the metadata database.

[0057] In the embodiment of the present invention, the control system of MongoDB determines the current analysis status of each MongoD instance by accessing the profile_status table in the metadata database.

[0058] Step 220: Enable the analysis function of the process whose current analysis status is enabled.

[0059] In one implementation, step 220 may specifically include:

[0060] For each of the processes whose current analysis status is enabled, the analysis function of the process is enabled by enabling the analyzer of the node where the process runs.

[0061] Specifically, if the current analysis status of the MongoD instance is enabled, it indicates that slow log collection is required for the MongoD instance. For MongoDB, the analysis function of the MongoD instance can record slow operations of all operation types (add, delete, modify, and query). Therefore, the slow log of the MongoD instance can be collected by enabling the analysis function of the MongoD instance.

[0062] After determining that the current analysis state of the MongoD instance is enabled, the control system of MongoDB can enable the analysis function of the MongoD instance. Specifically, the analysis function of the MongoD instance can be enabled by enabling the analyzer of the node where the MongoD instance runs.

[0063] The MongoD instance runs as a process in a docker container. The computing nodes in the form of physical machines, virtual machines, etc. deployed by the docker container running the MongoD instance can be understood as the nodes running the MongoD instance.

[0064] It should be noted that for each MongoD instance in the newly created MongoDB, the analysis function of each MongoD instance can be automatically enabled.

[0065] In an embodiment of the present invention, the control system of MongoDB enables the analysis function of the MongoD instance that needs to collect slow logs by enabling the analyzer of the node where the MongoD instance that needs to collect slow logs runs, thereby facilitating the collection of slow logs of the MongoD instance based on the analysis function of the MongoD instance that needs to collect slow logs when MongoDB is running.

[0066] Step 230: Collect the slow logs of each process with the analysis function enabled.

[0067] In one implementation, step 230 may specifically include:

[0068] Based on the analyzer of the node where the process runs, the slow log of the process is collected.

[0069] Specifically, the analysis function of the MongoD instance that needs to collect slow logs has been turned on. Therefore, when MongoDB is running, the control system of MongoDB can collect the slow logs generated during the operation of the MongoD instance that needs to collect slow logs based on the analysis function of the MongoD instance that needs to collect slow logs, and store the collected slow logs in the corresponding system.profile collection, thereby realizing the slow log collection of the MongoD instance that needs to collect slow logs.

[0070] The system.profile collection is a MongoDB fixed collection with a fixed capacity. When the collected slow logs use up the collection capacity, the subsequent collected slow logs will overwrite the slow logs first stored in the system.profile collection. In actual applications, the capacity of the system.profile collection can be 1M, and 1M of capacity can store 990 slow logs.

[0071] In the embodiment of the present invention, the slow logs of the MongoD instance that needs to collect the slow logs are collected based on the analysis function of the MongoD instance that needs to collect the slow logs.

[0072] Step 240: Store the slow log of each process based on the log file.

[0073] Specifically, the collected slow logs can be stored in log files. The collected slow logs are BSON format documents consisting of multiple fields. Therefore, each collected slow log can be converted into a string in JSON format and stored in a fixed log file slow-op.log, which can be stored on the disk of the MongoDB control system.

[0074] In actual applications, log files can also be pushed to a search server based on a full-text search engine (ElasticSearch, ES) for storage. The ES server can provide an API that supports slow log queries, so that the user terminal can query slow logs based on the API. Of course, ES can also provide slow log queries that support filtering conditions. For example, slow log queries that support filtering conditions such as date, resource identifier, and user identifier can be provided.

[0075] It should be noted that log files can be pushed to the ES server through the cloud log system.

[0076] In the embodiment of the present invention, the storage of the collected slow logs is achieved.

[0077] Step 250: Determine the target slow log in the log file according to the user identifier included in the received query request.

[0078] In one implementation, step 250 may specifically include:

[0079] According to the user identifier contained in the received query request, a target log file is determined in the log file; according to the current query timestamp and the query capacity, a current slow log is determined in the target log file; according to the current query timestamp and the query capacity, a next query timestamp is determined, and the next query timestamp is used as the current query timestamp, and the current slow log is determined in the target log file according to the current query timestamp and the query capacity, until the current slow log contains the latest slow log; all current slow logs determined in the target log file are determined as the target slow logs.

[0080] Further, determining a target log file in the log file according to the user identifier included in the received query request includes:

[0081] The log file is searched based on the user identifier included in the query request, and the target log file is determined according to the slow log in the log file that includes the user identifier.

[0082] The query capacity can be understood as the maximum amount of data that can be queried without affecting MongoDB performance.

[0083] Slow logs can be understood as BSON data containing multiple fields. The data format can be (instance ID, shard ID, node role, user library name, tsAnchor). Among them, the instance ID is the identifier of the MongoD instance corresponding to the slow log, for example, mongo-134nsgob38. When the MongoD instance is a replica set type instance, the shard ID is consistent with the instance ID. When the MongoD instance is a shard cluster type, the shard ID is the data shard identifier, for example, mongo-134nsgob38-shard1, mongo-134nsgob38-shard2.

[0084] The node role can be understood as the node on which the slow log corresponds to the MongoD instance running, for example: {Primary, Secondary, Secondary1, Secondary2, Secondary3, Secondary4, Secondary5}. In a sharded cluster type MongoD instance, each data shard itself is a replica set type MongoD instance. Therefore, the node roles can be recorded in the same way as the replica set type MongoD instance records the node roles. In a replica set type MongoD instance, multiple MongoDB nodes together form a replica set architecture of a primary node (Primary) and multiple slave nodes (Secondary). The number of nodes can be 3, 5, 7, etc. Each node corresponds to the Primary / Secondary role. When there are multiple Secondaries, a count suffix is ​​added to the system.

[0085] The user database name is the specific database created by the user on the MongoD instance, for example, test1, userDB2. tsAnchor is a timestamp location used to identify the collection progress.

[0086] Specifically, the ES server can be accessed based on the query request, and the target slow log can be determined in the log file stored in the ES server. The query request includes a user identifier. After the initial slow log is determined, the log file can be screened based on the user identifier, and the target log file can be determined according to the slow log containing the user tag in the log file. Specifically, the log file can be searched based on the user identifier contained in the query request, and the target log file can be determined according to the slow log containing the user identifier in the log file. Furthermore, the target slow log can be determined in the target log file. In the process of determining the target slow log, the current slow log can be determined in the target log file according to the current query timestamp and the query capacity that does not affect the performance of MongoDB. Specifically, the next query timestamp ts2 can be determined according to the current query timestamp ts1 and the query capacity tsStep, and multiple slow logs in the continuous time period [ts1, ts2] are batch collected in the target log file, and the collected multiple slow logs are determined as the current slow log. ts2>ts1, ts2-ts1=tsStep, the larger the tsStep is, the longer the query takes, and the more corresponding slow logs are collected.

[0087] If the current slow log contains the latest slow log, the current slow log is determined as the target slow log. If the current slow log does not contain the latest slow log, the next query timestamp is determined according to the current query timestamp and query capacity, and the next query timestamp is used as the current query timestamp. The current slow log is continued to be determined in the target log file according to the current query timestamp and query capacity until the current slow log contains the latest slow log. At this time, all current slow logs determined in the target log file can be determined as target slow logs, so that the slow log of the process created by the user corresponding to the user identifier contained in the query request is determined as the target slow log.

[0088] In the process of determining the target slow log, the current slow log determined each time may be stored in the cache, thereby implementing the caching of the slow log queried in real time.

[0089] In the embodiment of the present invention, it is achieved to determine the target slow log corresponding to the query request in the log file according to the received query request.

[0090] The control system of MongoDB consists of the monitoring module Monitor and the ES server. Monitor can be understood as a high-availability system composed of multiple nodes. Multiple nodes are deployed and run independently to form a high-availability mode. When the control system of MongoDB is running, only one node is in the running state as the master node to perform the control task of MongoDB, and the other nodes are in standby state as slave nodes. Once the current master node has an abnormal service, one of the slave nodes can be selected as the master node to continue to perform the control task of MongoDB.

[0091] In addition, MongoDB includes master nodes, slave nodes, and Hidden nodes. Normally, MongoDB runs based on master nodes and slave nodes. Hidden nodes are special replica set members used for tasks such as backup, data analysis, and testing. They do not participate in read and write requests and election processes, and are invisible to user terminals. After a master node and / or slave node fails, the Hidden node will be promoted to a slave node, and the original slave node will be promoted to a master node. MongoDB can continue to run based on the newly promoted master and slave nodes, thus ensuring the operation of MongoDB.

[0092] In the process of switching the master node of the MongoDB control system and / or the master node of MongoDB, the control tasks performed by the original master node need to continue to run, and the new master node needs to obtain the running status information of the original master node in order to continue to execute the control tasks. In the slow log collection task, the running status information can be understood as the last query timestamp.

[0093] Furthermore, the method also includes: after determining the change of the master node of the control system of MongoDB, and / or determining the change of the master node of MongoDB, determining the current query timestamp by comparing the last query timestamp and the latest timestamp of the slow log in the cache, and determining the target slow log in the log file based on the current query timestamp. Specifically, the last query timestamp and the latest timestamp of the slow log in the cache can be compared, and the timestamp closest to the current actual timestamp can be determined as the current query timestamp. Furthermore, the target slow log can be determined in the ES server based on the current query timestamp, avoiding data interruption of the queried target slow log and ensuring the data integrity of the target slow log.

[0094] Of course, when continuing to query the target slow log, you can also use a timestamp later than the latest timestamp of the slow log in the cache as the current query timestamp to query the target slow log. Although this will cause data redundancy, it can avoid data loss in the target slow log.

[0095] In actual applications, after determining the target slow log, the target slow log can be sent to the user terminal for the user to view and analyze, and the slow operations in the target slow log can be optimized to improve efficiency.

[0096] The slow log collection method provided by the embodiment of the present invention includes: determining the current analysis status of each process in the distributed document storage database; turning on the analysis function of the process whose current analysis status is turned on; collecting the slow logs of each process with the analysis function turned on; storing the slow logs of each process based on the log file; and determining the target slow log in the log file according to the user identifier contained in the received query request. In the above technical scheme, the control system of MongoDB can determine the current analysis status of each MongoD instance in MongoDB by accessing the profile_status table in the metadata library. After determining the current analysis status of each MongoD instance, the analysis function of each MongoD instance that needs to collect slow logs is turned on by turning on the analyzer of the node where each MongoD instance that needs to collect slow logs runs. When MongoDB is running, the slow logs of each MongoD instance are collected based on the analysis function of each MongoD instance that needs to collect slow logs, and the slow logs of each MongoD instance are stored based on the log file to realize the storage of the collected slow logs. After receiving the query request, Then, according to the user identifier contained in the query request, the target log file is determined in the log file, and the current slow log is determined in the target log file according to the current query timestamp and the query capacity, and the next query timestamp is determined according to the current query timestamp and the query capacity, and the next query timestamp is used as the current query timestamp, and the execution is returned to determine the current slow log in the target log file according to the current query timestamp and the query capacity until the current slow log contains the latest slow log, and all current slow logs determined in the target log file are determined as the target slow logs, so as to realize the query of the slow logs, without turning on the analysis function of all MongoD instances in MongoDB, to realize the collection and query of the slow logs, and reduce the impact on the performance of MongoDB.

[0097] Figure 3 This is a schematic diagram of the structure of a slow log collection device provided by an embodiment of the present invention. The device and the slow log collection method of the above embodiments belong to the same inventive concept, and the details not described in detail in the embodiment of the slow log collection device can refer to the embodiment of the above slow log collection method.

[0098] The specific structure of the slow log collection device is as follows Figure 3 As shown, including:

[0099] The start module 310 is used to determine the current analysis status of each process in the distributed document storage database, and start the analysis function of the process whose current analysis status is start;

[0100] A collection module 320, used for collecting the slow logs of each process with the analysis function enabled, and storing the slow logs of each process based on log files;

[0101] The determination module 330 is used to determine a target slow log in the log file according to the received query request.

[0102] Based on the above embodiment, the start module 310 is specifically used for:

[0103] The current analysis status of each process is determined based on the received trigger operation for each process and the previous analysis status of each process; for each process whose current analysis status is turned on, the analysis function of the process is turned on by turning on the analyzer of the node where the process runs.

[0104] Based on the above embodiment, the acquisition module 320 is specifically used for:

[0105] Based on an analyzer of a node where the process runs, slow logs of the process are collected; and based on a log file, the slow logs of each process are stored.

[0106] Based on the above embodiment, the determination module 330 is specifically used for:

[0107] The target slow log is determined in the log file according to the user identifier included in the received query request.

[0108] In one implementation, determining the target slow log in the log file according to the user identifier included in the received query request includes:

[0109] According to the user identifier contained in the received query request, a target log file is determined in the log file; according to the current query timestamp and the query capacity, a current slow log is determined in the target log file; according to the current query timestamp and the query capacity, a next query timestamp is determined, and the next query timestamp is used as the current query timestamp, and the current slow log is determined in the target log file according to the current query timestamp and the query capacity, until the current slow log contains the latest slow log; all current slow logs determined in the target log file are determined as the target slow logs.

[0110] In one implementation, determining a target log file in the log file according to a user identifier included in the received query request includes:

[0111] The log file is searched based on the user identifier included in the query request, and the target log file is determined according to the slow log in the log file that includes the user identifier.

[0112] The slow log collection device provided in the embodiment of the present invention can execute the slow log collection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the slow log collection method.

[0113] It is worth noting that in the embodiment of the above-mentioned slow log collection device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be realized; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0114] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 4 A block diagram of an exemplary electronic device 4 suitable for implementing embodiments of the present invention is shown. Figure 4 The electronic device 4 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0115] like Figure 4 As shown, the electronic device 4 is in the form of a general purpose computing electronic device. The components of the electronic device 4 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting different system components (including the system memory 28 and the processing unit 16).

[0116] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0117] The electronic device 4 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 4, including volatile and non-volatile media, removable and non-removable media.

[0118] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The electronic device 4 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 4 not shown, usually called a "hard drive"). Although Figure 4Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The system memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0119] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0120] The electronic device 4 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), one or more devices that enable a user to interact with the electronic device 4, and / or any device that enables the electronic device 4 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the electronic device 4 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 20. Figure 4 As shown, the network adapter 20 communicates with other modules of the electronic device 4 via the bus 18. It should be understood that although Figure 4 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 4, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0121] The processing unit 16 executes various functional applications and page displays by running the program stored in the system memory 28, for example, implementing the slow log collection method provided in the embodiment of the present invention, which includes:

[0122] Determine the current analysis status of each process in the distributed document storage database, and enable the analysis function of the process whose current analysis status is enabled;

[0123] Collect the slow log of each process with the analysis function enabled, and store the slow log of each process based on the log file;

[0124] A target slow log is determined in the log file according to the received query request.

[0125] Of course, those skilled in the art can understand that the processor can also implement the technical solution of the slow log collection method provided by any embodiment of the present invention.

[0126] An embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, for example, a slow log collection method provided by an embodiment of the present invention is implemented. The method includes:

[0127] Determine the current analysis status of each process in the distributed document storage database, and enable the analysis function of the process whose current analysis status is enabled;

[0128] Collect the slow log of each process with the analysis function enabled, and store the slow log of each process based on the log file;

[0129] A target slow log is determined in the log file according to the received query request.

[0130] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0131] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0132] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0133] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0134] It should be understood by those skilled in the art that the modules or steps of the present invention described above can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0135] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A slow log collection method, characterized in that: include: Determine the current analysis status of each process in the distributed document storage database, and enable the analysis function of the process whose current analysis status is enabled; Collect the slow log of each process with the analysis function enabled, and store the slow log of each process based on the log file; A target slow log is determined in the log file according to the received query request.

2. The slow log collection method according to claim 1, characterized in that: Determine the current analysis status of each process in the distributed document store database, including: The current analysis status of each process is determined according to the received trigger operation for each process and the previous analysis status of each process.

3. The slow log collection method according to claim 1, characterized in that: The analysis function of the process whose current analysis status is enabled is enabled, including: For each process whose current analysis status is enabled, the analysis function of the process is enabled by enabling the analyzer of the node where the process runs.

4. The slow log collection method according to claim 3, characterized in that: Collect the slow logs of each process with the analysis function enabled, including: Based on the analyzer of the node where the process runs, the slow log of the process is collected.

5. The slow log collection method according to claim 1, characterized in that: Determining a target slow log in the log file according to the received query request includes: The target slow log is determined in the log file according to the user identifier included in the received query request.

6. The slow log collection method according to claim 5, characterized in that: Determining a target slow log in the log file according to a user identifier included in the received query request includes: Determining a target log file in the log file according to a user identifier included in the received query request; Determine the current slow log in the target log file according to the current query timestamp and the query capacity; Determine a next query timestamp according to the current query timestamp and the query capacity, and use the next query timestamp as the current query timestamp, and return to execute determining a current slow log in the target log file according to the current query timestamp and the query capacity until the current slow log contains the latest slow log; All current slow logs determined in the target log file are determined as the target slow logs.

7. The slow log collection method according to claim 6, characterized in that: Determining a target log file in the log file according to a user identifier included in the received query request includes: The log file is searched based on the user identifier included in the query request, and the target log file is determined according to the slow log in the log file that includes the user identifier.

8. A slow log collection device, characterized in that: include: An enabling module, used to determine the current analysis status of each process in the distributed document storage database, and enable the analysis function of the process whose current analysis status is enabled; A collection module, used for collecting the slow logs of each process with the analysis function enabled, and storing the slow logs of each process based on log files; The determination module is used to determine the target slow log in the log file according to the received query request.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the slow log collection method as described in any one of claims 1 to 7.

10. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions are used to execute the slow log collection method as described in any one of claims 1 to 7 when executed by a computer processor.

Images