Performance data storage methods, apparatus, devices and storage media

Abstract

This invention provides a performance data storage method, apparatus, device, and storage medium. In response to a data acquisition task instruction for a monitored object, it collects object performance data and interface performance data of the monitored object; in response to receiving interface performance data, it stores the interface performance data in a first database and updates the corresponding statistical data of the interface performance indicators in real time; in response to receiving object performance data, it stores the object performance data in a second database; the object performance data in the second database is stored in regions according to object performance indicators; in response to a data statistics request for a first object, it determines the corresponding target performance indicator; it retrieves the corresponding statistical data of the indicators from the first database, or determines the statistical data of the indicators from the target storage region in the second database by determining the performance data corresponding to the target performance indicator. This method improves the response efficiency of performance data queries and reduces the database load.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a performance data storage method, apparatus, device, and storage medium. Background Technology

[0002] Stress testing of large-scale software systems is a core component in ensuring the stable operation of system services after launch. During stress testing, massive amounts of multi-type monitoring data are continuously generated, placing extremely high demands on data storage and processing efficiency. Current technologies often use document-based databases such as MongoDB to store stress testing data and calculate and display corresponding statistical results in real time when receiving client query requests. However, when facing large-scale, high-concurrency stress testing scenarios, the database needs to withstand high-frequency writes of massive amounts of data, making it highly susceptible to database crashes due to disk overload. Furthermore, data statistical analysis requires scanning the entire dataset for real-time calculations, which is not only inefficient but also further exacerbates the database load. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a performance data storage method, apparatus, device and storage medium to reduce the read and write pressure of database.

[0004] In a first aspect, embodiments of the present invention provide a performance data storage method, the method comprising: in response to a data acquisition task instruction for a monitored object, acquiring object performance data and interface performance data of the monitored object; in response to receiving interface performance data, storing the interface performance data in a first database, and determining, in real time, statistical data of the indicators corresponding to the interface performance indicators of the monitored object in the first database based on the interface performance data; in response to receiving object performance data, storing the object performance data in a second database; wherein, in the second database, the object performance data is stored in regions according to the object performance indicators of the monitored object; in response to a data statistics request for a first object in the monitored object, determining the target performance indicator to be statistically analyzed corresponding to the data statistics request; and based on the target performance indicator, obtaining statistical data of the indicators corresponding to the target performance indicator of the first object from the first database, or determining the performance data corresponding to the target performance indicator in a target storage region of the second database, and determining statistical data of the indicators corresponding to the target performance indicator of the first object based on the performance data in the target storage region.

[0005] Secondly, embodiments of the present invention provide a performance data storage device, comprising: a first acquisition module, configured to acquire object performance data and interface performance data of the monitored object in response to a data acquisition task instruction for the monitored object; a first update module, configured to store the interface performance data in a first database in response to receiving the interface performance data, and update the statistical data of the interface performance indicators corresponding to the monitored object in the first database in real time based on the interface performance data; a first storage module, configured to store the object performance data in a second database in response to receiving the object performance data; wherein, in the second database, the object performance data is stored in regions according to the object performance indicators of the monitored object; and a first determination module, configured to determine the target performance indicator to be statistically analyzed corresponding to the data statistical request for a first object in the monitored object; and, based on the target performance indicator, obtain the statistical data of the target performance indicator corresponding to the first object from the first database, or determine the performance data corresponding to the target performance indicator in a target storage region of the second database, and determine the statistical data of the target performance indicator corresponding to the first object based on the performance data in the target storage region.

[0006] Thirdly, embodiments of the present invention provide an electronic device, including a processor and a memory, wherein the memory stores machine-executable instructions that can be executed by the processor, and the processor executes the machine-executable instructions to implement the above-described performance data storage method.

[0007] Fourthly, embodiments of the present invention provide a storage medium storing machine-executable instructions. When the machine-executable instructions are invoked and executed by a processor, the machine-executable instructions cause the processor to implement the aforementioned performance data storage method.

[0008] The embodiments of the present invention bring the following beneficial effects: The aforementioned performance data storage method, apparatus, device, and storage medium include: in response to a data acquisition task instruction for a monitored object, collecting object performance data and interface performance data of the monitored object; in response to receiving interface performance data, storing the interface performance data in a first database, and updating the statistical data corresponding to the interface performance indicators of the monitored object in the first database in real time based on the interface performance data; in response to receiving object performance data, storing the object performance data in a second database; wherein, in the second database, the object performance data is stored in regions according to the object performance indicators of the monitored object; in response to a data statistics request for a first object among the monitored objects, determining the target performance indicator to be statistically analyzed corresponding to the data statistics request; and based on the target performance indicator, obtaining the statistical data corresponding to the target performance indicator of the first object from the first database, or determining the performance data corresponding to the target performance indicator in a target storage region of the second database, and determining the statistical data corresponding to the target performance indicator of the first object based on the performance data in the target storage region.

[0009] In this approach, the interface performance data of the monitored object is stored in a first database, while the object performance data is stored in a second database. The first database focuses on the storage and real-time statistics of interface performance data. During the interface performance data writing phase, statistical data corresponding to the interface performance metrics of the monitored object is generated synchronously, avoiding the statistical latency and excessive resource consumption caused by calculating statistical data after receiving data statistics requests. In the second database, object performance data is stored independently in separate areas according to the object performance metrics of the monitored object, physically isolating performance data corresponding to different object performance metrics to avoid query redundancy and disk overhead caused by mixed storage of performance data for different object performance metrics. When a statistical request for the first object is received, the target performance metric to be statistically analyzed for the first object is first determined. If the target performance metric is an interface performance metric, the corresponding statistical data is directly extracted from the first database; if the target performance metric is an object performance metric, the performance data corresponding to the target performance metric of the first object is directly extracted from the dedicated storage area corresponding to that object performance metric, and the corresponding statistical data is calculated. This avoids traversing and scanning the performance data corresponding to all object performance metrics of the first object, significantly reducing database resource consumption. This approach improves the response efficiency of performance data queries and reduces the load on the database.

[0010] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained in accordance with the structures particularly pointed out in the description, claims and drawings.

[0011] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0012] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0013] Figure 1 A flowchart illustrating a performance data storage method provided in an embodiment of the present invention; Figure 2 An architecture for a performance data storage system provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of a performance data storage device provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] During stress testing, massive amounts of diverse monitoring data are continuously generated. Current technologies often employ a single storage architecture based on document-oriented databases such as MongoDB to uniformly process various stress test data. Its workflow primarily encompasses three layers: acquisition, storage, and processing. The acquisition layer directly connects to the node under test via the Spy component to synchronously collect data; the storage layer writes the stress test data uniformly to the MongoDB database; and the processing layer, when users view reports or expert systems request data, initiates queries to MongoDB via the backend service, performing aggregation operations such as average calculation and sorting in real time during the query process.

[0016] This approach works fine in small-scale load testing scenarios, but performance and availability issues become apparent when facing large-scale, high-concurrency load testing tasks. MongoDB's ability to handle massive time-series data writing and aggregation analysis is insufficient. Massive data writing leads to a surge in disk pressure, frequent slow logs that can easily cause database crashes, and low data compression rates result in wasted disk space. Furthermore, data statistical analysis requires scanning the entire dataset for real-time calculations, which is not only inefficient but also further exacerbates the database load.

[0017] Based on this, embodiments of the present invention provide a performance data storage method, apparatus, electronic device, and storage medium, which can be applied to performance data storage scenarios.

[0018] To facilitate understanding of this embodiment, a performance data storage method disclosed in this invention will first be described in detail, such as... Figure 1 As shown, this method includes the following steps: Step S102: In response to the data acquisition task instruction for the monitored object, collect the object performance data and interface performance data of the monitored object.

[0019] The monitored objects mentioned above can be servers, covering various deployment forms such as physical machines and container services. The aforementioned object performance data refers to a type of performance data generated during the operation of the monitored object and its internal software processes under load testing scenarios. This type of performance data is the actual resource consumption and running status results collected according to preset object performance indicators. The object performance indicators can specifically include server overall CPU load, memory utilization, disk read / write I / O throughput, network bandwidth utilization, as well as multiple indicators such as resource consumption, process survival status, number of threads, and process CPU utilization of the software processes inside the server.

[0020] The aforementioned interface performance data refers to a type of performance data generated during the operation of the business function interfaces of the monitored object. This type of performance data is the actual collection result obtained under load testing scenarios for the business function interfaces of the monitored object, according to preset interface performance indicators. The interface performance indicators specifically include multiple metrics such as the response time of the business function interface, the number of times the business function interface is called, the number of callbacks to the business function interface, and the number of failed calls to the business function interface.

[0021] Upon receiving a data collection task instruction for the monitored object, the Spy component can be responsible for collecting various performance data of the monitored object. This Spy component is the data collection component of the load testing platform's collection layer, capable of directly establishing a connection with the server being tested to achieve synchronous collection of object performance data and interface performance data.

[0022] To decouple data acquisition from storage, the Spy component in this invention no longer directly interfaces with the database. Instead, it encapsulates all collected performance data into standardized message packets of a unified format and sends them to the Darkhouse data receiving service via an asynchronous communication mechanism. The Darkhouse data receiving service is a backend service component that acts as an intermediate routing layer for data flow. It parses the message packets and categorizes and routes the parsed data. Specifically, it forwards the parsed interface performance data to a first database cache to support high-frequency real-time queries, and forwards the parsed object performance data to a second database, thereby ensuring efficient data flow and hierarchical storage.

[0023] In one approach, the Darkhouse data receiving service can be deployed on a Kubernetes container cluster, maintaining the same stateless characteristics as the load testing engine. The Darkhouse data receiving service receives performance data in a unified format asynchronously sent by the Spy component and then forwards the performance data to the corresponding database. When the data collection volume surges, the Kubernetes container cluster can automatically add service instances, and when the collection tasks decrease, it can reduce the number of instances, ensuring no data loss while conserving server resources.

[0024] Step S104: In response to receiving interface performance data, store the interface performance data in the first database, and determine the statistical data of the interface performance indicators corresponding to the monitored object in the first database based on the interface performance data.

[0025] The aforementioned first database can be deployed independently or in a distributed cluster environment, such as a Redis-TimeSeries database or a Redis-TimeSeries cluster, to meet the high-concurrency writing and real-time statistical requirements of interface performance data. The aforementioned interface performance metrics refer to the monitoring items set for the business function interfaces of the monitored objects. These metrics specifically include the response time of the business function interface, the number of times the business function interface is called, the number of callbacks to the business function interface, and the number of failed calls to the business function interface, among other indicators. The performance data corresponding to each interface performance metric refers to the specific monitoring results for that particular metric. It can be understood that each data collection for a business function interface will generate performance data corresponding to each of the aforementioned interface performance metrics for that specific business function interface.

[0026] For the received interface performance data, the first database establishes a corresponding performance data sequence for each interface performance metric of the business interface to which the performance data belongs. This sequence is specifically used to store the performance data corresponding to that interface performance metric. Simultaneously, it acquires the attribute information corresponding to the interface performance data, such as data owner information and collection task identifier. The performance data sequence is then associated with the attribute information to obtain the performance data sequences corresponding to multiple interface performance metrics under that attribute information. This allows for better classification, storage, refined management, and rapid retrieval of interface performance data of different dimensions and types. Within the performance data sequence, each performance data point is arranged in a pre-defined chronological order.

[0027] The process of constructing a performance data sequence is as follows: First, extract the attribute information corresponding to the interface performance data, including data owner information, collection task identifier, and interface identifier of the business function interface to which the interface performance data belongs; then, based on the above attribute information and the received interface performance data, construct a unique corresponding performance data sequence for each interface performance indicator of the business function interface to which the interface performance data belongs, and establish the association between the performance data sequence and the attribute information.

[0028] The specific association method can be as follows: generate a unique sequence identifier based on the attribute information corresponding to the interface performance data, use this sequence identifier as the exclusive identifier of the corresponding performance data sequence, associate it with the performance data sequence, realize the precise binding of performance data sequence and attribute information, and ensure the uniqueness and traceability of performance data sequence.

[0029] For example, the received interface performance data includes: the response time t1 of the business function interface, the number of times the business function interface is called c1, the number of callbacks to the business function interface h1, and the number of failed calls to the business function interface s1. The attribute information corresponding to the interface performance data is determined as: data owner information A1, collection task identifier T1, and the interface identifier D1 of the business function interface. Based on the interface performance data, the interface performance indicators are determined to include: the number of times the business function interface is called, the number of callbacks to the business function interface, the number of failed calls to the business function interface, and the response time of the business function interface. Based on the attribute information corresponding to each interface performance indicator and the interface performance data, four sequence identifiers are established: "A1-T1-D1-Number of Calls", "A1-T1-D1-Number of Calls", "A1-T1-D1-Number of Failed Calls", and "A1-T1-D1-Response Time".

[0030] Based on each sequence identifier established above, the first database synchronously creates a unique corresponding performance data sequence. Each performance data sequence is specifically used to store performance data under the same interface performance metric dimension indicated by the corresponding sequence identifier, and the sequence identifier is associated with the corresponding performance data sequence. This ensures that when interface performance data with the same attribute information is received again, the corresponding sequence identifier can be matched with the attribute information and interface performance metric, thereby quickly locating the performance data sequence associated with the sequence identifier and directly appending the performance data corresponding to the interface performance metric to the corresponding performance data sequence, completing the update of the performance data sequence and the storage of the interface performance data.

[0031] The performance data in the performance data sequence is arranged in a preset time order. That is, each performance data sequence stores only all time-series performance data of a specific interface performance indicator under the same data owner, the same collection task, and the same business function interface. For example, the performance data sequence corresponding to "A1-T1-D1-Response Duration" can be [(Collection Timestamp 1, Response Duration 1), (Collection Timestamp 2, Response Duration 2), (Collection Timestamp 3, Response Duration 3)...] After determining the performance data sequence corresponding to the interface performance indicators under this attribute information, the statistical data of each interface performance indicator under this attribute information is simultaneously determined based on the performance data sequence. The statistical data of indicators is obtained by statistically analyzing the performance data in the performance data sequence. For example, for the performance data sequence corresponding to a certain interface performance indicator, the maximum value, minimum value, average value, sum, variance, etc. of these performance data can be obtained by statistically calculating the performance data, which are used as the statistical data of indicators corresponding to that interface performance indicator.

[0032] Subsequently, each time interface performance data with the same attribute information is received, while incrementally writing the data into the corresponding performance data sequence, the statistical data of the interface performance metrics corresponding to that performance data sequence is updated in real time based on the historical data and the newly added data. This ensures that the statistical data of the metrics is consistent with the latest performance data sequence, meeting the real-time requirements of subsequent data statistics requests. This ensures that subsequent data statistics requests can directly obtain results, significantly improving response efficiency.

[0033] Step S106: In response to receiving object performance data, store the object performance data in the second database; wherein, in the second database, the object performance data is stored in different regions according to the object performance indicators of the monitored object.

[0034] In the aforementioned second database, object performance data can be stored in different regions according to the object performance metrics of the monitored objects. As a preferred example, the second database can be a columnar database called ClickHouse, which has pre-built dedicated data storage tables containing metric columns that correspond one-to-one with multiple preset performance metrics. Each metric column corresponds to a designated storage region in the second database.

[0035] By writing the performance data corresponding to each object's performance metric in the object performance data into the metric column corresponding to that object performance metric, the object performance data can be classified and stored in the designated storage area corresponding to each metric column, thereby achieving the storage goal of managing object performance data by metric by region.

[0036] This storage method physically isolates the performance data corresponding to different object performance metrics. When querying the performance metrics data of a specific object later, the corresponding storage area can be located directly without scanning the entire table data, which greatly reduces query redundancy and I / O overhead and improves data reading and statistical efficiency.

[0037] Step S108: Respond to the data statistics request for the first object among the monitored objects, determine the target performance index to be statistically analyzed corresponding to the data statistics request; and based on the target performance index, obtain the indicator statistics data corresponding to the target performance index of the first object from the first database, or determine the performance data corresponding to the target performance index in the target storage area of ​​the second database, and determine the indicator statistics data corresponding to the target performance index of the first object based on the performance data in the target storage area.

[0038] The first object can be a business function interface of the monitored object, the monitored object itself, or a software process running on the monitored object.

[0039] The aforementioned target performance metrics specifically refer to the performance metrics corresponding to the first object, which can be directly parsed and determined from the data statistics request. In addition to containing the target performance metrics to be statistically analyzed for the first object, the data statistics request may also carry parameters to limit the statistical scope in some scenarios, such as specifying the performance data corresponding to the target performance metrics generated within a certain collection period. Simultaneously, the request can also include parameters such as sorting method and statistical method to support diverse statistical needs. For example, the data statistics request could be specifically "query the 10 processes with the highest CPU usage on January 1st" or "statistically analyze the minimum response time of the first business function interface," achieving customized statistics by carrying corresponding parameters.

[0040] When a statistics request for the first object is received, the target performance metric of the first object can be located first. If the target performance metric is an interface performance metric, the metric statistics data corresponding to the target performance metric of the first object can be directly extracted from the first database without additional calculation.

[0041] If the target performance metric is an interface performance metric, the statistical data corresponding to the first object is directly extracted from the first database. If the performance metric is an object performance metric, the performance data corresponding to the first object is directly extracted from the dedicated storage area corresponding to the object performance metric, and the corresponding statistical data is calculated. This avoids traversing and scanning the performance data corresponding to all performance metrics of the first object, significantly reducing the resource consumption overhead of the database. The above method improves the response efficiency of performance data queries and reduces the load pressure on the database.

[0042] The aforementioned performance data storage method, apparatus, device, and storage medium include: in response to a data acquisition task instruction for a monitored object, collecting object performance data and interface performance data of the monitored object; in response to receiving interface performance data, storing the interface performance data in a first database, and updating the statistical data corresponding to the interface performance indicators of the monitored object in the first database in real time based on the interface performance data; in response to receiving object performance data, storing the object performance data in a second database; wherein, in the second database, the object performance data is stored in regions according to the object performance indicators of the monitored object; in response to a data statistics request for a first object among the monitored objects, determining the target performance indicator to be statistically analyzed corresponding to the data statistics request; and based on the target performance indicator, obtaining the statistical data corresponding to the target performance indicator of the first object from the first database, or determining the performance data corresponding to the target performance indicator in a target storage region of the second database, and determining the statistical data corresponding to the target performance indicator of the first object based on the performance data in the target storage region.

[0043] In this approach, the interface performance data of the monitored object is stored in a first database, while the object performance data is stored in a second database. The first database focuses on the storage and real-time statistics of interface performance data. During the interface performance data writing phase, statistical data corresponding to the interface performance metrics of the monitored object is generated synchronously, avoiding the statistical latency and excessive resource consumption caused by calculating statistical data after receiving data statistics requests. In the second database, object performance data is stored independently in separate areas according to the object performance metrics of the monitored object, physically isolating performance data corresponding to different object performance metrics to avoid query redundancy and disk overhead caused by mixed storage of performance data for different object performance metrics. When a statistical request for the first object is received, the target performance metric to be statistically analyzed for the first object is first determined. If the target performance metric is an interface performance metric, the corresponding statistical data is directly extracted from the first database; if the target performance metric is an object performance metric, the performance data corresponding to the target performance metric of the first object is directly extracted from the dedicated storage area corresponding to that object performance metric, and the corresponding statistical data is calculated. This avoids traversing and scanning the performance data corresponding to all object performance metrics of the first object, significantly reducing database resource consumption. This approach improves the response efficiency of performance data queries and reduces the load on the database.

[0044] In one approach, the interface performance data includes: performance data generated by business function interfaces in the monitored object; in response to receiving the first interface performance data, determining the first attribute information corresponding to the first interface performance data; wherein the first attribute information includes: data owner information of the first interface performance data, collection task identifier of the first interface performance data, and interface identifier of the first business function interface to which the first interface performance data belongs; in a first database, constructing a first performance data sequence corresponding to multiple interface performance indicators under the first attribute information; wherein the first performance data sequence is associated with the first attribute information; the performance data corresponding to the interface performance indicators in the first performance data sequence are arranged in a preset time order.

[0045] The aforementioned first attribute information includes: the data owner information of the interface performance data, the task identifier for collecting the interface performance data, and the interface identifier of the first business function interface in the first interface performance data.

[0046] In one approach, the received interface performance data includes: the response time t1 of the business function interface, the number of times the business function interface is called c1, the number of callbacks to the business function interface h1, and the number of failed calls to the business function interface s1. The attribute information corresponding to the interface performance data is determined as: data owner information A1, collection task identifier T1, and the interface identifier D1 of the business function interface. Based on the interface performance data, the interface performance metrics are determined to include: the number of times the business function interface is called, the number of callbacks to the business function interface, the number of failed calls to the business function interface, and the response time of the business function interface. Based on the attribute information corresponding to each interface performance metric and the interface performance data, four sequence identifiers are established: "A1-T1-D1-Number of Calls", "A1-T1-D1-Number of Calls", "A1-T1-D1-Number of Failed Calls", and "A1-T1-D1-Response Time".

[0047] Based on each sequence identifier established above, the first database synchronously creates a unique corresponding performance data sequence. Each performance data sequence is specifically used to store performance data under the same performance metric dimension indicated by the corresponding sequence identifier. The sequence identifier is associated with the corresponding performance data sequence to obtain the first performance data sequences corresponding to multiple interface performance metrics under the first attribute information. The performance data in the performance data sequence is arranged in a preset time order. That is, each performance data sequence only stores all time-series performance data of a specific performance metric under the same data owner, the same collection task, and the same business function interface. For example, the performance data sequence corresponding to "A1-T1-D1-Response Duration" can be [(Collection Timestamp 1, Response Duration 1), (Collection Timestamp 2, Response Duration 2), (Collection Timestamp 3, Response Duration 3)...]. Collection Timestamp 1 is earlier than Collection Timestamp 2, Collection Timestamp 2 is earlier than Collection Timestamp 3, and so on.

[0048] After determining the performance data sequence corresponding to the interface performance indicators under this attribute information, based on the performance data sequence, the statistical data of the indicators corresponding to each interface performance indicator under this attribute information are synchronously determined and updated in real time.

[0049] Specifically, for each interface performance indicator under the first attribute information, based on the first performance data sequence corresponding to the interface performance indicator, the first indicator statistical data corresponding to the interface performance indicator under the first attribute information is determined in the first database; in response to receiving the second interface performance data, the second attribute information corresponding to the second interface performance data is determined; based on the second attribute information and the first attribute information, the indicator statistical data corresponding to the interface performance indicator of the monitored object is updated in real time.

[0050] The statistical data for the indicator is obtained by statistically analyzing the performance data in the performance data sequence corresponding to the interface performance indicator. For example, if the interface performance indicator under the first attribute information is the number of times the first business function interface is called, then the corresponding statistical data for the first indicator can be the result corresponding to the total number of times the first business function interface is called; if the interface performance indicator is the response time of the first business function interface, then the statistical data for the first indicator can include: the total response time of the first business function interface, the minimum response time of the first business function interface, the longest response time of the first business function interface, the median response time of the interface, the average response time of the first business function interface, and the values ​​corresponding to multiple preset quantile times such as P50, P90, P95, P99, etc.

[0051] In response to receiving the second interface performance data, determine the second attribute information corresponding to the second interface performance data; based on the second attribute information and the first attribute information, update the indicator statistics corresponding to the interface performance indicators of the monitored object in real time.

[0052] Specifically, it is determined whether the second attribute information is the same as the first attribute information; if the second attribute information is the same as the first attribute information, based on the second interface performance data, the first performance data sequences corresponding to multiple interface performance indicators under the first attribute information are updated respectively; based on the updated first performance data sequences, the first indicator statistical data corresponding to each interface performance indicator is updated in the first database; if the second attribute information is different from the first attribute information, based on the second interface performance data and the second attribute information, the second performance data sequences corresponding to multiple interface performance indicators under the second attribute information are constructed in the first database respectively; wherein, the second performance data sequences are associated with the second attribute information; for each interface performance indicator under the second attribute information, based on the second performance data sequence corresponding to the interface performance indicator, the second indicator statistical data corresponding to the interface performance indicator under the second attribute information is determined in the first database.

[0053] In other words, if the second attribute information matches the first attribute information, the second interface performance data is incrementally written into the corresponding first performance data sequence. Specifically, the corresponding sequence identifier can be matched with the attribute information and the interface performance index, thereby quickly locating the performance data sequence associated with the sequence identifier and directly appending the performance data corresponding to the interface performance index to the corresponding performance data sequence.

[0054] Then, based on the updated first performance data sequence, the first indicator statistics corresponding to the interface performance indicators are updated in real time to ensure that the indicator statistics are consistent with the latest performance data sequence.

[0055] If the second attribute information is inconsistent with the first attribute information, according to the same construction rules as the first performance data sequence, an independent second performance data sequence is created for each interface performance indicator under the second attribute information. The relationship between the second performance data sequence and the second attribute information is clarified. Then, based on the performance data in each second performance data sequence, the second indicator statistics corresponding to each interface performance indicator are completed, and the second indicator statistics corresponding to each interface performance indicator under the second attribute information are obtained.

[0056] In practical implementation, if the first database is a Redis-TimeSeries database, its natively supported atomic commands can be fully utilized to complete the above update operation: On the one hand, the TS.ADD atomic command is used to write the performance data corresponding to each interface performance metric in the second interface performance data into the existing first performance data sequence, ensuring the atomicity and orderliness of data writing in high-concurrency scenarios and avoiding data overwriting or loss. On the other hand, the updated sequence data is read using TS.RANGE, and the first metric statistics are synchronously updated using Redis's atomic calculation commands. No additional locking mechanism is required throughout the process, which not only ensures data consistency but also significantly improves the efficiency of sequence updates and statistics refresh in high-concurrency scenarios, adapting to the real-time processing requirements of large-scale stress test data.

[0057] In one approach, the second database has a pre-configured data storage table; the data storage table contains multiple preset performance indicators, each corresponding to an indicator column; the indicator columns correspond to a designated storage area of ​​the second database. In response to receiving first object performance data, a first object performance indicator corresponding to the first object performance data is determined; the performance data corresponding to the first object performance indicator in the first object performance data is written into the first indicator column, so that the performance data corresponding to the first object performance indicator is stored in the designated storage area corresponding to the first indicator column; wherein, the first object performance indicator matches the preset performance indicator corresponding to the first indicator column.

[0058] In other words, the second database is pre-configured with a data storage table. This data storage table sets corresponding indicator columns for various preset object performance indicators (such as CPU load, memory utilization, disk I / O throughput, etc.), and each indicator column is mapped to a dedicated designated storage area within the database.

[0059] When performance data of a certain object is received, taking the performance data of the first object as an example, the first object performance index corresponding to the performance data of the object is first determined; then the first index column in the storage table that is consistent with the performance index of the first object is matched, and the performance data corresponding to the performance index of the first object in the performance data of the first object is written into the column, so that the performance data is automatically stored in the specified storage area associated with the first index column.

[0060] It should be noted that before writing the performance data corresponding to the first object performance index in the first object performance data into the first index column, the process also includes: determining the second object performance index that does not match the first object performance index in the preset performance index; and writing the index column corresponding to the second object performance index into the specified data.

[0061] In other words, before writing the performance data corresponding to the first object's performance metric in the first object's performance data into the matching first metric column, a supplementary write operation needs to be performed on the second object's performance metric in the data storage table that does not match the first object's performance metric, filling its corresponding metric column with specified data. This specified data is usually a default placeholder (such as -1), and its core function is to ensure the structural integrity and field consistency of each record in the data storage table, avoiding table structure chaos caused by missing fields in a single record, and completely avoiding the additional system overhead caused by frequent adjustments to the table structure. This ensures that subsequent batch queries, cross-metric correlation analysis, data export, and other operations on the full table data will not result in field misalignment or parsing failures.

[0062] Taking the object performance data storage of the game server under stress test as an example: if a certain collection only obtains the performance data corresponding to two types of indicators, namely CPU load and memory utilization, while the data storage table also contains indicators such as disk I / O throughput and network bandwidth utilization, then the specified data needs to be written into the indicator columns corresponding to these two types of uncollected object performance indicators in the data storage table.

[0063] In one approach, the data storage table contains a preset sorting rule; the sorting rule is used to indicate the sorting order of object performance data in the data storage table; the object attribute information corresponding to the first object performance data is determined; wherein, the object attribute information includes: the data owner information of the first object performance data, the acquisition task identifier of the first object performance data, the unit identifier of the monitoring unit of the monitored object, and the acquisition time of the first object performance data; the type of monitoring unit includes: device type and process type; based on the sorting rule and the object attribute information, the sorting order of the first object performance data in the data storage table is determined; according to the sorting order, the performance data corresponding to the first object performance index in the first object performance data is written into the first index column.

[0064] The aforementioned monitoring unit refers to the smallest independent carrier that performs performance data collection on the monitored object. Taking a server as an example, if the monitoring unit is a device type, then the monitoring unit corresponds to the entire machine device of the server being stress-tested, and its unit identifier is the server's device identifier. If the monitoring unit is a process type, then the monitoring unit corresponds to the service process running inside the server device, and its unit identifier can be a combination of the device identifier and the process identifier.

[0065] In other words, the data storage tables in the second database have pre-configured sorting rules that define the sorting logic for object performance data within the tables. In actual implementation, the data storage table structure can be designed as a wide table, containing a common collection time column, a unit identifier column, and hundreds of indicator columns.

[0066] When writing the performance data of the first object, the object attribute information corresponding to the data is first extracted, including: data owner information, collection task identifier, unit identifier of the monitoring unit corresponding to the monitoring object, and collection time. Then, according to the preset sorting rules of the data storage table, combined with the extracted object attribute information, the specific sorting position of the performance data of the first object in the table is determined; finally, according to the sorting position, the performance data corresponding to the performance index of the first object in the performance data of the first object is accurately written into the matching first index column.

[0067] Specifically, the sorting rules include one or more of the following: object performance data corresponding to the same data owner information are arranged consecutively in the data storage table; object performance data corresponding to the same acquisition task identifier are arranged consecutively in the data storage table; object performance data corresponding to the same type of monitoring unit are arranged consecutively in the data storage table; and object performance data are sorted according to the acquisition time order of the object performance data.

[0068] In other words, the preset sorting rules of the data storage table can be flexibly combined with one or more of the above sorting dimensions to achieve structured and orderly storage of object performance data.

[0069] In one specific implementation, the sorting logic of object performance data can be executed in the following hierarchical manner: First, all object performance data belonging to the same data owner are collected into the same initial data set; on this basis, object performance data with the same collection task identifier are further filtered out from this set and aggregated into a subset; then, within this subset, performance data records of the same monitoring unit type are centrally classified; finally, these records are arranged in an orderly manner according to the order of collection time.

[0070] When the second database is a ClickHouse database, the ORDER BY key for the data storage table can be explicitly set to (TenantID, TaskID, EntityType, Timestamp) when building the data storage table based on ClickHouse. Here, TenantID indicates the data owner information, TaskID is the task identifier, EntityType corresponds to the monitoring unit type, and Timestamp corresponds to the collection time. When object performance data is stored in the ClickHouse database, ClickHouse sorts the data and organizes its physical storage according to this key combination: first, it aggregates all data with the same TenantID; then, it aggregates data with the same TaskID within the same TenantID; next, it centrally stores records with the same EntityType within the task dimension; finally, it arranges the object performance data in order by Timestamp; and finally, ClickHouse stores the data column-wise and compresses it.

[0071] Ultimately, this allows for the continuous storage of object performance data for the same type of monitoring unit under the same tenant and the same data collection task at the physical storage level. By combining the characteristics of ClickHouse columnar storage and data partitioning, when querying the performance data corresponding to the target performance indicator, the system can directly locate the continuous physical storage blocks in the second database without cross-block scanning, which greatly reduces the number of disk I / O operations and significantly improves the query efficiency of large-scale performance data.

[0072] In one approach, when the target performance metric is an interface performance metric, statistical data of the metric corresponding to the interface performance metric of the first object are obtained from the first database.

[0073] In other words, when the target performance metric to be statistically analyzed is an interface performance metric, its target storage area can be directly located in the first database, and the statistical data of the interface performance metric corresponding to the first object, which is pre-calculated in real time based on the performance data sequence, can be extracted from the first database without additional calculation process, thus achieving rapid acquisition of interface performance metric statistical results.

[0074] Specifically, based on the interface identifier, data owner information, and collection task identifier of the first object, the attribute information of the interface performance data of the first object can be determined, the statistical data of the indicators associated with the attribute information can be determined, and the statistical data of the indicators corresponding to the interface performance indicators of the first object can be found from the statistical data of the indicators corresponding to multiple interface performance indicators.

[0075] In one approach, the target performance metrics include the object performance metrics of a first object. The data statistics request includes: a sorting request for the performance data corresponding to the object performance metrics of the first object; determining the sorting method and sorting quantity from the sorting request; determining the performance data corresponding to the object performance metrics of the first object in the target storage area of ​​the second database; obtaining the performance data corresponding to the object performance metrics of the first object from the target storage area; sorting the performance data corresponding to the performance metrics according to the sorting method; and extracting the performance data matching the sorting quantity from the sorting result to determine the metric statistics data.

[0076] In other words, when the target performance metric to be statistically analyzed is the object performance metric of the first object, and the data statistics request carries a sorting requirement, the data statistics request is first parsed to clarify the specific sorting method, such as arranging the performance data corresponding to the object performance metric in ascending or descending order, and the number of sorting data, such as taking the top 10 performance data or the bottom 5 performance data.

[0077] Then, locate the dedicated storage area corresponding to the object performance index of the first object, extract the performance data corresponding to the object performance index from it, and sort the performance data according to the parsed sorting method.

[0078] Finally, data matching the sorting quantity is selected from the sorting results and used as statistical data to meet the customized sorting and statistical needs of object performance indicators.

[0079] In another approach, the data statistics request includes: an aggregation statistics request for the performance data corresponding to the object performance index of the first object. This type of data statistics request is used to perform multi-dimensional numerical aggregation calculations on the performance data, such as calculating the average, maximum, minimum, sum, and preset quantile values ​​of the performance data corresponding to the target performance index.

[0080] For example, if the interface performance metric of the first object is the CPU utilization rate of a game server process, and the data statistics request is "to collect statistics on the average, maximum, and P95 percentile CPU utilization rate of server device D1 belonging to party A1 under task T1 on January 1st", then the Spy component parses the core parameters in the request: data owner (A1), collection task identifier (T1), monitoring unit identifier, time range (January 1st), and aggregation statistics type (average, maximum, P95 percentile). Subsequently, the Spy component generates a precise SQL query statement for the ClickHouse database based on these parameters, and queries the indicator column data corresponding to the CPU utilization rate of the process in the second database.

[0081] When ClickHouse receives a query request corresponding to an SQL query, it uses the vectorized execution engine to perform aggregation calculations on the wide table data within that time range. Since the performance data is already sorted by (TenantID, TaskID, EntityType, Timestamp) and stored in columnar format, it only needs to read the storage area corresponding to the CPU utilization index column, without scanning other irrelevant areas, greatly reducing disk I / O overhead. At the same time, it calls native aggregation functions (AVG() to calculate the average, MAX() to calculate the maximum, and quantile() to calculate the P95 quantile) to complete multi-dimensional statistics and obtain the index statistics corresponding to the CPU utilization index of that process.

[0082] In one approach, a preset data acquisition component collects object performance data and interface performance data of the monitored object; the collected object performance data and interface performance data are encapsulated into a message packet of a specified format and sent asynchronously to a preset backend service component; in response to the backend service component receiving the message packet, the message packet is parsed, and the parsed interface performance data is stored in a first database; in response to the backend service component receiving the message packet, the message packet is parsed, and the parsed object performance data is stored in a second database.

[0083] The specified format mentioned above refers to a predefined unified message format, such as JSON or XML. Regardless of whether the collected data is object performance data or interface performance data, it is encapsulated according to the same message structure so that the backend service components can uniformly receive, parse, and classify it.

[0084] The aforementioned preset data acquisition component can be a Spy component, which is the data acquisition component of the load testing platform's acquisition layer. This Spy component can directly establish a connection with the server being tested, enabling synchronous acquisition of object performance data and interface performance data. The aforementioned backend service component can be the Darkhouse data receiving service, which acts as an intermediate routing layer for data flow. It parses message packets and categorizes and routes the parsed data.

[0085] To decouple data acquisition from storage, the Spy component in this invention no longer directly interfaces with the database. Instead, it encapsulates all collected performance data into standardized message packets of a unified format and sends them to the Darkhouse data receiving service via an asynchronous communication mechanism. This Darkhouse data receiving service, acting as an intermediate routing layer for data flow, parses the message packets and categorizes and routes the parsed data according to data type. Specifically, it forwards the parsed interface performance data to a first database cache to support high-frequency real-time queries, and forwards the parsed object performance data to a second database, thereby ensuring layered data storage.

[0086] In one approach, in response to a data acquisition task instruction for a monitored object, metadata of the data acquisition task is acquired and stored in a third database; wherein, the metadata includes: attribute information of the monitored object and configuration information corresponding to the data acquisition task.

[0087] The attribute information of the monitored objects includes: the unit identifier of the monitoring unit of the monitored object, such as the device identifier of the server, the service process identifier running inside the server device, etc. The configuration information corresponding to the data acquisition task can be the basic attributes and execution rules of the acquisition task, such as the acquisition task identifier, task start time, task end time, data acquisition frequency, etc. The third database can be a document-oriented database such as MongoDB, which is used to store the configuration information corresponding to the data acquisition task and the attribute information of the monitored object's performance data. Compared to relational databases, MongoDB's document-oriented storage structure is more suitable for the flexible expansion needs of configuration information. When the acquisition task configuration needs to add fields, data can be written directly without adjusting the database table structure, which can better adapt to the dynamic changes in business needs.

[0088] In other words, after receiving a data collection task instruction for the monitored object, the Spy component will also collect the metadata of the data collection task, along with the object's performance data and the interface's performance data, and encapsulate them into a message packet. After the backend service component parses the message packet, it stores the parsed metadata in a third-party database, thereby providing basic metadata support for the subsequent classification, storage, attribute matching, metric statistics, and fast querying of interface performance data.

[0089] In one implementation, upon receiving a data collection task instruction for the monitored object, the Spy component collects the object's performance data, interface performance data, and metadata of the data collection task. The Spy component then encapsulates this collected data into a unified format message packet and asynchronously sends it to the Darkhouse data receiving service. The Darkhouse data receiving service categorizes and routes the received data: metadata is routed to the third database MongoDB; interface performance data is routed to the first database Redis-TimeSeries cluster; and object performance data is batch-routed to the second database ClickHouse cluster.

[0090] In one approach, configuration information corresponding to the data collection task is obtained from a pre-set third database, and the configuration information and the statistical data of the target performance indicators are determined as the response result of the data statistics request.

[0091] The configuration information corresponding to the above-mentioned data collection tasks can be the basic attributes and execution rules of the data collection tasks, such as data collection task identifier, task start time, task end time, data collection frequency, etc.

[0092] The aforementioned third database can be a document-oriented database such as MongoDB. This MongoDB database is used to store configuration information corresponding to data acquisition tasks and attribute information of the performance data of monitored objects. Compared to relational databases, MongoDB's document-oriented storage structure is more suitable for the flexible expansion needs of configuration information. When the acquisition task configuration requires the addition of new fields, data can be written directly without adjusting the database table structure, thus better adapting to dynamic changes in business requirements.

[0093] In other words, after determining the statistical data of the indicators, the configuration information corresponding to the data collection task can be obtained from the preset third database, and the configuration information and the statistical data of the indicators corresponding to the target performance indicators can be determined as the response result of the data statistics request.

[0094] In one approach, in response to the completion of the data acquisition task, the interface performance data is stored in a second database, wherein the interface performance data is stored in the second database according to the interface performance indicators of the monitored object in different regions.

[0095] Once the data collection task is completed, the raw interface performance data will be stored in a second database, such as a columnar database like ClickHouse. Within this second database, the interface performance data will be stored in separate regions according to the different interface performance metrics of the monitored objects, laying the foundation for efficient subsequent queries.

[0096] In one approach, both the first database and the second database are deployed in a distributed cluster, which employs a sharding and replication mechanism. According to a preset sharding rule, the interface performance data is distributed to the first shard node in the first database cluster and synchronously stored in the replica node corresponding to the first shard node. According to a preset sharding rule, the object performance data is distributed to the second database shard node in the second database cluster and synchronously stored in the replica node corresponding to the second database shard node.

[0097] The aforementioned "first database cluster" refers to the distributed cluster containing the first database, and the aforementioned "second database cluster" refers to the distributed cluster containing the second database. The aforementioned preset sharding rule can be a sharding rule based on data ownership information, meaning that performance data of monitored objects with the same ownership are fixedly distributed to the same database sharding node. Sharding nodes are related to the preset sharding rule. For example, when the preset sharding rule is a sharding rule based on data ownership information, the aforementioned "first sharding node" refers to an independent storage node in the first database cluster specifically designed to carry interface performance data with attribute information of a specific data ownership information. The replica node corresponding to the aforementioned "first sharding node" refers to a backup storage node in the first database cluster that is paired with the first sharding node and used to synchronously store the same interface performance data. Similarly, the aforementioned "second sharding node" refers to an independent storage node in the second database cluster specifically designed to carry object performance data with attribute information of a specific data ownership information. The replica node corresponding to the aforementioned "second sharding node" refers to a backup storage node in the second database cluster that is paired with the second sharding node and used to synchronously store the same object performance data.

[0098] In other words, both the first and second databases are deployed in a distributed cluster architecture, and the cluster ensures high availability, high scalability, and efficient access to data through a dual mechanism of sharding and replication. Specifically, for interface performance data, it is distributed to the corresponding first shard node in the first database cluster for storage, and at the same time, the batch of interface performance data is synchronized in real time to the replica node pre-paired to the first shard node to achieve consistency between primary and secondary data; For object performance data, according to the same preset sharding rules, it is distributed to the corresponding second shard node in the second database cluster for storage, and synchronized to the replica node of the second shard node, thus completing the distributed master-slave deployment of the two types of performance data.

[0099] When a specified shard node fails, read and write requests for the specified shard node will be switched to the replica node corresponding to the specified shard node; wherein, the specified shard node includes: the first shard node and / or the second shard node.

[0100] In other words, when a shard node in the first or second database cluster fails, the corresponding database cluster will automatically and seamlessly switch read and write requests originally directed to the failed shard to its corresponding replica node. The entire process requires no manual intervention, ensuring the continuity and stability of the platform's external data services.

[0101] In a specific embodiment, such as Figure 2 This is a diagram of a hierarchical data flow architecture, where the Redis database is the first database mentioned earlier, and the ClickHouse database is the second database mentioned earlier. Figure 2 Loadlab's front end refers to the user-facing visual interactive interface, which is the direct entry point between the user and the system. It provides functions such as performance data query, task configuration, and result display. Users can initiate data statistics requests through the front end.

[0102] The Spy component is responsible for parsing requests from the Loadlab frontend or data analysis service, and making targeted calls to Redis, ClickHouse, or MongoDB based on the request type (such as real-time query, aggregate statistics, sorting); it also encapsulates the query results, integrates the statistical data with the task configuration information in MongoDB, and returns them to the requester. Figure 2 Metadata refers to the configuration information of the data collection task, the attribute information of the monitored object, etc.

[0103] Figure 2 Through a hierarchical storage and collaborative service architecture, the system achieves efficient collection, storage, and retrieval of performance data for monitored objects. First, the data acquisition script is responsible for collecting performance data and metadata in real time from monitored objects such as servers, processes, and interfaces, such as CPU utilization, memory usage, and interface response time.

[0104] Then, the interface performance data is first written to a Redis database for high-frequency real-time queries, ensuring low-latency access. Simultaneously, object performance data is written to a ClickHouse database for persistent storage of all data and complex analysis.

[0105] Then, the data collection service performs real-time statistics on the interface performance data from the Redis cache to obtain the corresponding statistical data of the interface performance metrics of the monitored object. After the data collection task is completed, the collected interface performance data is persistently stored in ClickHouse.

[0106] When a user initiates a data statistics request through the Loadlab frontend or data analysis service, the request is forwarded to the Spy component. The Spy component receives the request from the Loadlab frontend and analysis service, and intelligently selects the database to query the data based on the request type, such as real-time query or aggregate statistics.

[0107] The Spy component first parses parameters such as the target performance metric, time range, and sorting type from the data statistics request. If the data statistics request requires a task configuration context, Spy retrieves the corresponding collection task's configuration information, such as task ID and collection frequency, from MongoDB. Then, it selects the appropriate database based on the statistics request. Specifically, when a statistics request for the first object is received, it first locates the performance metric of the first object. If the performance metric is an interface performance metric, it directly extracts the corresponding metric statistics from that first database. If the performance metric is an object performance metric, it directly extracts the corresponding performance data from the dedicated storage area corresponding to that object's performance metric to calculate the metric statistics. Then, Spy integrates the statistical data and configuration information returned by the MongoDB database into a standardized response, which is then sent back to the Loadlab frontend or data analysis service. When the data collection task stops, the system persists the task's metadata to MongoDB to ensure task traceability.

[0108] In this architecture, the Darkhouse service and load testing engine can adopt a stateless design and be deployed via Kubernetes containerization, enabling automatic horizontal scaling (HPA) based on CPU / memory levels.

[0109] To decouple data collection and storage, the Spy component in this invention no longer directly connects to the database. Instead, it encapsulates all collected performance data into standardized message packets of a unified format and sends them to the Darkhouse data receiving service via an asynchronous communication mechanism. Here, both the Darkhouse service and the load testing engine adopt a stateless architecture and are deployed on a Kubernetes containerized platform. Under this architecture, service instances do not store local session or business data, and all instances are completely equivalent. Combined with Kubernetes' horizontal auto-scaling capabilities, the system can monitor the CPU and memory resource levels of containers in real time. When the load increases, it automatically adds container instances to share the request pressure; when the load decreases, it automatically reduces the number of instances to save resources, thereby achieving a dynamic balance between service throughput and resource utilization.

[0110] For the corresponding method embodiments described above, see [link to relevant documentation]. Figure 3 The diagram shows a performance data storage device, which includes: The first acquisition module 302 is used to respond to the data acquisition task instruction for the monitored object and acquire the object performance data and interface performance data of the monitored object. The first update module 304 is used to respond to the received interface performance data, store the interface performance data in the first database, and update the statistical data of the interface performance indicators of the monitored object in the first database in real time based on the interface performance data. The first storage module 306 is used to store the object performance data to the second database in response to receiving the object performance data; wherein, in the second database, the object performance data is stored in different regions according to the object performance indicators of the monitored object. The first determining module 308 is used to respond to a data statistics request for a first object among the monitored objects, determine the target performance index to be statistically analyzed corresponding to the data statistics request, and obtain the indicator statistics data corresponding to the target performance index of the first object from the first database based on the target performance index, or determine the performance data corresponding to the target performance index in the target storage area of ​​the second database, and determine the indicator statistics data corresponding to the target performance index of the first object based on the performance data in the target storage area.

[0111] In this approach, the interface performance data of the monitored object is stored in a first database, while the object performance data is stored in a second database. The first database focuses on the storage and real-time statistics of interface performance data. During the interface performance data writing phase, statistical data corresponding to the interface performance metrics of the monitored object is generated synchronously, avoiding the statistical latency and excessive resource consumption caused by calculating statistical data after receiving data statistics requests. In the second database, object performance data is stored independently in separate areas according to the object performance metrics of the monitored object, physically isolating performance data corresponding to different object performance metrics to avoid query redundancy and disk overhead caused by mixed storage of performance data for different object performance metrics. When a statistical request for the first object is received, the target performance metric to be statistically analyzed for the first object is first determined. If the target performance metric is an interface performance metric, the corresponding statistical data is directly extracted from the first database; if the target performance metric is an object performance metric, the performance data corresponding to the target performance metric of the first object is directly extracted from the dedicated storage area corresponding to that object performance metric, and the corresponding statistical data is calculated. This avoids traversing and scanning the performance data corresponding to all object performance metrics of the first object, significantly reducing database resource consumption. This approach improves the response efficiency of performance data queries and reduces the load on the database.

[0112] The aforementioned interface performance data includes: performance data corresponding to the business function interfaces of the monitored object; The aforementioned first update module is used to, in response to receiving the first interface performance data, determine the first attribute information corresponding to the first interface performance data; wherein, the first attribute information includes: the data owner information of the first interface performance data, the collection task identifier of the first interface performance data, and the interface identifier of the first business function interface to which the first interface performance data belongs; in the first database, a first performance data sequence corresponding to multiple interface performance indicators under the first attribute information is constructed; wherein, the first performance data sequence is associated with the first attribute information; the performance data corresponding to the interface performance indicators in the first performance data sequence are arranged in a preset time order.

[0113] The aforementioned first update module is used to determine, for each interface performance indicator under the first attribute information, the first indicator statistical data corresponding to the interface performance indicator under the first attribute information in the first database based on the first performance data sequence corresponding to the interface performance indicator; in response to receiving the second interface performance data, determine the second attribute information corresponding to the second interface performance data; and based on the second attribute information and the first attribute information, update the indicator statistical data corresponding to the interface performance indicators of the monitored object in real time.

[0114] The aforementioned first update module is used to determine whether the second attribute information is the same as the first attribute information; if the second attribute information is the same as the first attribute information, based on the second interface performance data, update the first performance data sequences corresponding to multiple interface performance indicators under the first attribute information; based on the updated first performance data sequences, update the first indicator statistical data corresponding to each interface performance indicator in the first database; if the second attribute information is different from the first attribute information, based on the second interface performance data and the second attribute information, construct the second performance data sequences corresponding to multiple interface performance indicators under the second attribute information in the first database; wherein, the second performance data sequences are associated with the second attribute information; for each interface performance indicator under the second attribute information, based on the second performance data sequence corresponding to the interface performance indicator, determine the second indicator statistical data corresponding to the interface performance indicator under the second attribute information in the first database.

[0115] The aforementioned second database has a pre-set data storage table; the data storage table contains multiple indicator columns corresponding to preset performance indicators; the indicator columns correspond to a designated storage area of ​​the second database; the first storage module is used to, in response to receiving the first object performance data, determine the first object performance indicator corresponding to the first object performance data; write the performance data corresponding to the first object performance indicator in the first object performance data into the first indicator column, so that the performance data corresponding to the first object performance indicator is stored in the designated storage area corresponding to the first indicator column; wherein, the preset performance indicator corresponding to the first indicator column matches the first object performance indicator.

[0116] The aforementioned device further includes a first writing module, used to determine a second object performance index that does not match the first object performance index from the preset performance indexes; and to write the index column corresponding to the second object performance index into specified data.

[0117] The aforementioned data storage table contains preset sorting rules; the sorting rules are used to indicate the sorting order of object performance data in the data storage table; the first writing module is used to determine the object attribute information corresponding to the first object performance data; wherein, the object attribute information includes: the data owner information of the first object performance data, the acquisition task identifier of the first object performance data, the unit identifier of the monitoring unit of the monitored object, and the acquisition time of the first object performance data; the type of monitoring unit includes: device type and process type; based on the sorting rules and object attribute information, the sorting order of the first object performance data in the data storage table is determined; according to the sorting order, the performance data corresponding to the first object performance index in the first object performance data is written into the first index column.

[0118] The sorting rules mentioned above include one or more of the following: object performance data corresponding to the same data owner information are arranged consecutively in the data storage table; object performance data corresponding to the same acquisition task identifier are arranged consecutively in the data storage table; object performance data corresponding to the same type of monitoring unit are arranged consecutively in the data storage table; object performance data are sorted according to the acquisition time order of object performance data.

[0119] The aforementioned first determining module is used to obtain statistical data of the interface performance index corresponding to the first object from the first database when the target performance index is an interface performance index.

[0120] The aforementioned target performance metrics include the object performance metrics of the first object, and the aforementioned data statistics request includes: a sorting request for the performance data corresponding to the object performance metrics of the first object; the aforementioned first determining module is used to determine the performance data corresponding to the object performance metrics of the first object in the target storage area of ​​the second database, obtain the performance data corresponding to the object performance metrics of the first object from the target storage area, sort the performance data according to the sorting method, and extract the performance data matching the sorting quantity from the sorting result to determine the metric statistics data.

[0121] The first acquisition module is used to acquire object performance data and interface performance data of the monitored object through a preset data acquisition component; encapsulate the acquired object performance data and interface performance data into a message packet of a specified format, and send it to the preset backend service component asynchronously.

[0122] The first update module is used to respond to the message packet received by the backend service component, parse the message packet, and store the parsed interface performance data in the first database; The first storage module is used to respond to the message packet received by the backend service component, parse the message packet, and store the parsed object performance data in the second database.

[0123] The second storage module is used to collect metadata of the data acquisition task and store the metadata in the third database; the metadata includes: attribute information of the monitored object and configuration information corresponding to the data acquisition task.

[0124] The aforementioned device includes a second determining module, used to obtain configuration information corresponding to the data acquisition task from a preset third database, and determine the configuration information and the statistical data of the target performance indicators as the response result of the data statistics request.

[0125] The aforementioned device includes a second storage module for storing interface performance data into a second database in response to the completion of a data acquisition task. In the second database, the interface performance data is stored in different regions according to the interface performance indicators of the monitored object.

[0126] Both the first database and the first database are deployed in a distributed cluster, which adopts a sharding and replication mechanism. The first storage module is used to distribute interface performance data to the first shard node in the first database cluster according to a preset sharding rule, and synchronously store it to the replica node corresponding to the first shard node. The first storage module is used to distribute object performance data to the second shard node in the second database cluster according to a preset sharding rule, and synchronously store it to the replica node corresponding to the second shard node.

[0127] The aforementioned device includes a first switching module, used to switch read and write requests for a specified shard node to the replica node corresponding to the specified shard node when the specified shard node experiences a failure; wherein the specified shard node includes: a first shard node and / or a second shard node.

[0128] This embodiment also provides an electronic device, including a processor and a memory. The memory stores machine-executable instructions that can be executed by the processor, and the processor executes the machine-executable instructions to implement the above-described performance data storage method. This electronic device can be a server or a terminal device.

[0129] See Figure 4 As shown, the electronic device includes a processor 100 and a memory 101. The memory 101 stores machine-executable instructions that can be executed by the processor 100. The processor 100 executes the machine-executable instructions to implement the above-described performance data storage method.

[0130] Furthermore, Figure 4The electronic device shown also includes a bus 102 and a communication interface 103, with the processor 100, the communication interface 103 and the memory 101 connected via the bus 102.

[0131] The memory 101 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 103 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc. The bus 102 may be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0132] Processor 100 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of processor 100 or by instructions in software form. Processor 100 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software module can reside in a readily available storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 101, and the processor 100 reads the information from memory 101 and, in conjunction with its hardware, completes the steps of the method described in the foregoing embodiments.

[0133] The processor in the aforementioned electronic device, by executing machine-executable instructions, can implement the following operations of the aforementioned performance data storage method: In response to a data acquisition task instruction for a monitored object, it acquires object performance data and interface performance data of the monitored object; in response to receiving interface performance data, it stores the interface performance data in a first database, and updates the statistical data corresponding to the interface performance indicators of the monitored object in the first database in real time based on the interface performance data; in response to receiving object performance data, it stores the object performance data in a second database; wherein, in the second database, the object performance data is stored in regions according to the object performance indicators of the monitored object; in response to a data statistics request for a first object among the monitored objects, it determines the target performance indicator to be statistically analyzed corresponding to the data statistics request; and based on the target performance indicator, it retrieves the statistical data corresponding to the target performance indicator of the first object from the first database, or determines the performance data corresponding to the target performance indicator in the target storage area of ​​the second database, and determines the statistical data corresponding to the target performance indicator of the first object based on the performance data in the target storage area.

[0134] In this approach, the interface performance data of the monitored object is stored in a first database, while the object performance data is stored in a second database. The first database focuses on the storage and real-time statistics of interface performance data. During the interface performance data writing phase, statistical data corresponding to the interface performance metrics of the monitored object is generated synchronously, avoiding the statistical latency and excessive resource consumption caused by calculating statistical data after receiving data statistics requests. In the second database, object performance data is stored independently in separate areas according to the object performance metrics of the monitored object, physically isolating performance data corresponding to different object performance metrics to avoid query redundancy and disk overhead caused by mixed storage of performance data for different object performance metrics. When a statistical request for the first object is received, the target performance metric to be statistically analyzed for the first object is first determined. If the target performance metric is an interface performance metric, the corresponding statistical data is directly extracted from the first database; if the target performance metric is an object performance metric, the performance data corresponding to the target performance metric of the first object is directly extracted from the dedicated storage area corresponding to that object performance metric, and the corresponding statistical data is calculated. This avoids traversing and scanning the performance data corresponding to all object performance metrics of the first object, significantly reducing database resource consumption. This approach improves the response efficiency of performance data queries and reduces the load on the database.

[0135] The aforementioned interface performance data includes: performance data corresponding to the business function interfaces of the monitored object; The processor in the aforementioned electronic device, by executing machine-executable instructions, can implement the following operations of the aforementioned performance data storage method: in response to receiving first interface performance data, determining first attribute information corresponding to the first interface performance data; wherein, the first attribute information includes: data owner information of the first interface performance data, acquisition task identifier of the first interface performance data, and interface identifier of the first business function interface to which the first interface performance data belongs; in a first database, constructing first performance data sequences corresponding to multiple interface performance indicators under the first attribute information; wherein, the first performance data sequences are associated with the first attribute information; the performance data corresponding to the interface performance indicators in the first performance data sequences are arranged in a preset time order.

[0136] The processor in the aforementioned electronic device, by executing machine-executable instructions, can perform the following operations of the aforementioned performance data storage method: for each interface performance indicator under the first attribute information, based on the first performance data sequence corresponding to the interface performance indicator, determine the first indicator statistical data corresponding to the interface performance indicator under the first attribute information in the first database; in response to receiving the second interface performance data, determine the second attribute information corresponding to the second interface performance data; based on the second attribute information and the first attribute information, update the indicator statistical data corresponding to the interface performance indicator of the monitored object in real time.

[0137] The processor in the aforementioned electronic device, by executing machine-executable instructions, can perform the following operations of the aforementioned performance data storage method: determining whether the second attribute information is the same as the first attribute information; if the second attribute information is the same as the first attribute information, updating the first performance data sequences corresponding to multiple interface performance indicators under the first attribute information based on the second interface performance data; updating the first indicator statistical data corresponding to each interface performance indicator in the first database based on the updated first performance data sequences; if the second attribute information is different from the first attribute information, constructing the second performance data sequences corresponding to multiple interface performance indicators under the second attribute information in the first database based on the second interface performance data and the second attribute information; wherein the second performance data sequences are associated with the second attribute information; for each interface performance indicator under the second attribute information, determining the second indicator statistical data corresponding to the interface performance indicator under the second attribute information in the first database based on the second performance data sequence corresponding to the interface performance indicator.

[0138] The aforementioned second database has a pre-set data storage table; the data storage table contains multiple index columns corresponding to preset performance indicators; the index columns correspond to a designated storage area of ​​the second database; the processor in the aforementioned electronic device, by executing machine-executable instructions, can implement the following operations of the aforementioned performance data storage method: in response to receiving first object performance data, determining the first object performance indicator corresponding to the first object performance data; writing the performance data corresponding to the first object performance indicator in the first object performance data into the first index column, so that the performance data corresponding to the first object performance indicator is stored in the designated storage area corresponding to the first index column; wherein, the preset performance indicator corresponding to the first index column matches the first object performance indicator.

[0139] The processor in the aforementioned electronic device can perform the following operations of the aforementioned performance data storage method by executing machine-executable instructions: determining a second object performance index that does not match the first object performance index from the preset performance indexes; and writing the index column corresponding to the second object performance index into specified data.

[0140] The aforementioned data storage table contains preset sorting rules; the sorting rules are used to indicate the sorting order of object performance data in the data storage table; the processor in the aforementioned electronic device, by executing machine-executable instructions, can implement the following operations of the aforementioned performance data storage method: determining the object attribute information corresponding to the first object performance data; wherein, the object attribute information includes: the data owner information of the first object performance data, the acquisition task identifier of the first object performance data, the unit identifier of the monitoring unit of the monitored object, and the acquisition time of the first object performance data; the type of monitoring unit includes: device type and process type; determining the sorting order of the first object performance data in the data storage table based on the sorting rules and object attribute information; and writing the performance data corresponding to the first object performance index in the first object performance data into the first index column according to the sorting order.

[0141] The sorting rules mentioned above include one or more of the following: object performance data corresponding to the same data owner information are arranged consecutively in the data storage table; object performance data corresponding to the same acquisition task identifier are arranged consecutively in the data storage table; object performance data corresponding to the same type of monitoring unit are arranged consecutively in the data storage table; object performance data are sorted according to the acquisition time order of object performance data.

[0142] The processor in the aforementioned electronic device can perform the following operations of the aforementioned performance data storage method by executing machine-executable instructions: when the target performance indicator is an interface performance indicator, retrieve the statistical data of the indicator corresponding to the interface performance indicator of the first object from the first database.

[0143] The aforementioned target performance metrics include object performance metrics. The processor in the aforementioned electronic device, by executing machine-executable instructions, can implement the following operations of the aforementioned performance data storage method: determining the sorting method and sorting quantity from the sorting request; determining the performance data corresponding to the object performance metrics of the first object in the target storage area of ​​the second database; obtaining the performance data corresponding to the object performance metrics of the first object from the target storage area; sorting the performance data according to the sorting method; and extracting the performance data matching the sorting quantity from the sorting result to determine the metric statistics.

[0144] The processor in the aforementioned electronic device, by executing machine-executable instructions, can implement the following operations of the aforementioned performance data storage method: acquiring object performance data and interface performance data of the monitored object through a preset data acquisition component; encapsulating the acquired object performance data and interface performance data into a message packet of a specified format, and sending it asynchronously to a preset backend service component; in response to the backend service component receiving the message packet, parsing the message packet, and storing the parsed interface performance data in a first database; and in response to the backend service component receiving the message packet, parsing the message packet, and storing the parsed object performance data in a second database.

[0145] The processor in the aforementioned electronic device can perform the following operations of the aforementioned performance data storage method by executing machine-executable instructions: collecting metadata of the data acquisition task and storing the metadata in a third database; wherein, the metadata includes: attribute information of the monitored object and configuration information corresponding to the data acquisition task.

[0146] The processor in the aforementioned electronic device can perform the following operations of the aforementioned performance data storage method by executing machine-executable instructions: obtaining configuration information corresponding to the data acquisition task from a preset third database, and determining the configuration information and the statistical data of the target performance indicators as the response result of the data statistics request.

[0147] The processor in the aforementioned electronic device can perform the following operations of the aforementioned performance data storage method by executing machine-executable instructions: in response to the end of the data acquisition task, the interface performance data is stored in a second database, wherein the interface performance data is stored in the second database according to the interface performance indicators of the monitored object in different regions.

[0148] Both the first database and the first database are deployed in a distributed cluster, which adopts a sharding and replication mechanism. The processor in the electronic device can perform the following operations of the performance data storage method by executing machine-executable instructions: distributing interface performance data to the first shard node in the first database cluster according to a preset sharding rule, and synchronously storing it to the replica node corresponding to the first shard node; the first storage module is used to distribute object performance data to the second shard node in the second database cluster according to a preset sharding rule, and synchronously storing it to the replica node corresponding to the second shard node.

[0149] The processor in the aforementioned electronic device can perform the following operations of the aforementioned performance data storage method by executing machine-executable instructions: when a specified shard node fails, read and write requests for the specified shard node are switched to the replica node corresponding to the specified shard node; wherein the specified shard node includes: a first shard node and / or a second shard node.

[0150] This embodiment also provides a storage medium that stores machine-executable instructions. When the machine-executable instructions are called and executed by the processor, the machine-executable instructions cause the processor to implement the above-described performance data storage method.

[0151] The machine-executable instructions stored in the aforementioned storage medium can be executed to perform the following operations in the aforementioned performance data storage method: In response to a data acquisition task instruction for a monitored object, collect object performance data and interface performance data of the monitored object; in response to receiving interface performance data, store the interface performance data in a first database, and based on the interface performance data, update the statistical data corresponding to the interface performance indicators of the monitored object in the first database in real time; in response to receiving object performance data, store the object performance data in a second database; wherein, in the second database, the object performance data is stored in regions according to the object performance indicators of the monitored object; in response to a data statistics request for a first object among the monitored objects, determine the target performance indicator to be statistically analyzed corresponding to the data statistics request; and based on the target performance indicator, obtain the statistical data corresponding to the target performance indicator of the first object from the first database, or determine the performance data corresponding to the target performance indicator in the target storage area of ​​the second database, and determine the statistical data corresponding to the target performance indicator of the first object based on the performance data in the target storage area.

[0152] In this approach, the interface performance data of the monitored object is stored in a first database, while the object performance data is stored in a second database. The first database focuses on the storage and real-time statistics of interface performance data. During the interface performance data writing phase, statistical data corresponding to the interface performance metrics of the monitored object is generated synchronously, avoiding the statistical latency and excessive resource consumption caused by calculating statistical data after receiving data statistics requests. In the second database, object performance data is stored independently in separate areas according to the object performance metrics of the monitored object, physically isolating performance data corresponding to different object performance metrics to avoid query redundancy and disk overhead caused by mixed storage of performance data for different object performance metrics. When a statistical request for the first object is received, the target performance metric to be statistically analyzed for the first object is first determined. If the target performance metric is an interface performance metric, the corresponding statistical data is directly extracted from the first database; if the target performance metric is an object performance metric, the performance data corresponding to the target performance metric of the first object is directly extracted from the dedicated storage area corresponding to that object performance metric, and the corresponding statistical data is calculated. This avoids traversing and scanning the performance data corresponding to all object performance metrics of the first object, significantly reducing database resource consumption. This approach improves the response efficiency of performance data queries and reduces the load on the database.

[0153] The aforementioned interface performance data includes: performance data corresponding to the business function interfaces of the monitored object; The machine-executable instructions stored in the aforementioned storage medium can be executed to perform the following operations in the aforementioned performance data storage method: In response to receiving first interface performance data, determining the first attribute information corresponding to the first interface performance data; wherein, the first attribute information includes: data owner information of the first interface performance data, acquisition task identifier of the first interface performance data, and interface identifier of the first business function interface to which the first interface performance data belongs; in a first database, constructing a first performance data sequence corresponding to multiple interface performance indicators under the first attribute information; wherein, the first performance data sequence is associated with the first attribute information; the performance data corresponding to the interface performance indicators in the first performance data sequence are arranged in a preset time order.

[0154] The machine-executable instructions stored in the aforementioned storage medium can be executed to perform the following operations in the aforementioned performance data storage method: for each interface performance indicator under the first attribute information, based on the first performance data sequence corresponding to the interface performance indicator, determine the first indicator statistical data corresponding to the interface performance indicator under the first attribute information in the first database; in response to receiving the second interface performance data, determine the second attribute information corresponding to the second interface performance data; based on the second attribute information and the first attribute information, update the indicator statistical data corresponding to the interface performance indicator of the monitored object in real time.

[0155] The machine-executable instructions stored in the aforementioned storage medium can be executed to perform the following operations in the aforementioned performance data storage method: determining whether the second attribute information is the same as the first attribute information; if the second attribute information is the same as the first attribute information, updating the first performance data sequences corresponding to multiple interface performance indicators under the first attribute information based on the second interface performance data; updating the first indicator statistical data corresponding to each interface performance indicator in the first database based on the updated first performance data sequences; if the second attribute information is different from the first attribute information, constructing the second performance data sequences corresponding to multiple interface performance indicators under the second attribute information in the first database based on the second interface performance data and the second attribute information; wherein the second performance data sequences are associated with the second attribute information; for each interface performance indicator under the second attribute information, determining the second indicator statistical data corresponding to the interface performance indicator under the second attribute information in the first database based on the second performance data sequence corresponding to the interface performance indicator.

[0156] The aforementioned second database has a pre-set data storage table; the data storage table contains multiple index columns corresponding to preset performance indicators; the index columns correspond to a designated storage area of ​​the second database; the aforementioned storage medium stores machine-executable instructions, and by executing these machine-executable instructions, the following operations in the aforementioned performance data storage method can be implemented: in response to receiving first object performance data, determining the first object performance indicator corresponding to the first object performance data; writing the performance data corresponding to the first object performance indicator in the first object performance data into the first index column, so that the performance data corresponding to the first object performance indicator is stored in the designated storage area corresponding to the first index column; wherein, the preset performance indicator corresponding to the first index column matches the first object performance indicator.

[0157] The machine-executable instructions stored in the aforementioned storage medium can be used to perform the following operations in the aforementioned performance data storage method: determine a second object performance index that does not match the first object performance index from the preset performance indexes; and write the index column corresponding to the second object performance index into the specified data.

[0158] The aforementioned data storage table contains preset sorting rules; the sorting rules are used to indicate the sorting order of object performance data in the data storage table; the machine-executable instructions stored in the aforementioned storage medium can, by executing the machine-executable instructions, realize the following operations in the aforementioned performance data storage method: determining the object attribute information corresponding to the first object performance data; wherein, the object attribute information includes: the data owner information of the first object performance data, the acquisition task identifier of the first object performance data, the unit identifier of the monitoring unit of the monitored object, and the acquisition time of the first object performance data; the type of monitoring unit includes: device type and process type; determining the sorting order of the first object performance data in the data storage table based on the sorting rules and object attribute information; and writing the performance data corresponding to the first object performance index in the first object performance data into the first index column according to the sorting order.

[0159] The sorting rules mentioned above include one or more of the following: object performance data corresponding to the same data owner information are arranged consecutively in the data storage table; object performance data corresponding to the same acquisition task identifier are arranged consecutively in the data storage table; object performance data corresponding to the same type of monitoring unit are arranged consecutively in the data storage table; object performance data are sorted according to the acquisition time order of object performance data.

[0160] The machine-executable instructions stored in the aforementioned storage medium can be used to perform the following operations in the aforementioned performance data storage method: when the target performance indicator is an interface performance indicator, obtain the statistical data of the indicator corresponding to the interface performance indicator of the first object from the first database.

[0161] The machine-executable instructions stored in the aforementioned storage medium can be used to perform the following operations in the aforementioned performance data storage method: determining the sorting method and sorting quantity from the sorting request; determining the performance data corresponding to the object performance index of the first object in the target storage area of ​​the second database; obtaining the performance data corresponding to the object performance index of the first object from the target storage area; sorting the performance data according to the sorting method; and extracting the performance data matching the sorting quantity from the sorting result to determine the index statistics.

[0162] The machine-executable instructions stored in the aforementioned storage medium can be executed to perform the following operations in the aforementioned performance data storage method: Collecting object performance data and interface performance data of the monitored object through a preset data acquisition component; encapsulating the collected object performance data and interface performance data into a message packet of a specified format and sending it asynchronously to a preset backend service component; in response to the backend service component receiving the message packet, parsing the message packet, and storing the parsed interface performance data in a first database; in response to the backend service component receiving the message packet, parsing the message packet, and storing the parsed object performance data in a second database.

[0163] The machine-executable instructions stored in the aforementioned storage medium can be used to perform the following operations in the aforementioned performance data storage method: collecting metadata of the data acquisition task and storing the metadata in a third database; wherein, the metadata includes: attribute information of the monitored object and configuration information corresponding to the data acquisition task.

[0164] The machine-executable instructions stored in the aforementioned storage medium can be used to perform the following operations in the aforementioned performance data storage method: obtain the configuration information corresponding to the data acquisition task from a preset third database, and determine the configuration information and the statistical data of the target performance indicators as the response result of the data statistics request.

[0165] The machine-executable instructions stored in the aforementioned storage medium can be executed to perform the following operations in the aforementioned performance data storage method: in response to the end of the data acquisition task, the interface performance data is stored in the second database, wherein the interface performance data is stored in the second database according to the interface performance indicators of the monitored object in different regions.

[0166] Both the first database and the first database are deployed in a distributed cluster, which adopts a sharding and replication mechanism. The machine-executable instructions stored in the storage medium can perform the following operations in the performance data storage method by executing the machine-executable instructions: distributing interface performance data to the first shard node in the first database cluster according to the preset sharding rules, and synchronously storing it to the replica node corresponding to the first shard node; the first storage module is used to distribute object performance data to the second shard node in the second database cluster according to the preset sharding rules, and synchronously storing it to the replica node corresponding to the second shard node.

[0167] The machine-executable instructions stored in the aforementioned storage medium can be executed to achieve the following operations in the aforementioned performance data storage method: when a specified shard node fails, read and write requests for the specified shard node are switched to the replica node corresponding to the specified shard node; wherein, the specified shard node includes: a first shard node and / or a second shard node.

[0168] The computer program products of the performance data storage method, apparatus, electronic device and storage medium provided in the embodiments of the present invention include a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods in the preceding method embodiments. For specific implementation, please refer to the method embodiments, which will not be repeated here.

[0169] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system and apparatus described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0170] Furthermore, in the description of the embodiments of the present invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in the present invention based on the specific circumstances.

[0171] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0172] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0173] Finally, it should be noted that the above embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A performance data storage method, characterized in that, The method includes: In response to a data collection task instruction for a monitored object, collect the object performance data and interface performance data of the monitored object; In response to receiving the interface performance data, the interface performance data is stored in a first database, and based on the interface performance data, the statistical data of the interface performance indicators of the monitored object is updated in real time in the first database. In response to receiving the object performance data, the object performance data is stored in a second database; wherein, in the second database, the object performance data is stored in different regions according to the object performance indicators of the monitored object; In response to a data statistics request for a first object among the monitored objects, the system determines the target performance metric to be statistically analyzed corresponding to the data statistics request; and based on the target performance metric, it retrieves the metric statistics data corresponding to the target performance metric of the first object from the first database, or determines that the performance data corresponding to the target performance metric is in the target storage area of ​​the second database, and determines the metric statistics data corresponding to the target performance metric of the first object based on the performance data in the target storage area.

2. The method according to claim 1, characterized in that, The interface performance data includes: performance data generated by the business function interfaces in the monitored object; The step of storing the interface performance data in a first database in response to receiving the interface performance data includes: In response to receiving the first interface performance data, the system determines the first attribute information corresponding to the first interface performance data; wherein, the first attribute information includes: the data owner information of the first interface performance data, the collection task identifier of the first interface performance data, and the interface identifier of the first business function interface to which the first interface performance data belongs. In the first database, a first performance data sequence is constructed corresponding to multiple interface performance indicators under the first attribute information; wherein, the first performance data sequence is associated with the first attribute information; and the performance data corresponding to the interface performance indicators in the first performance data sequence are arranged in a preset time order.

3. The method according to claim 2, characterized in that, The step of updating the statistical data of the interface performance metrics of the monitored object in real time in the first database based on the interface performance data includes: For each interface performance metric under the first attribute information, based on the first performance data sequence corresponding to the interface performance metric, the first indicator statistical data corresponding to the interface performance metric under the first attribute information is determined in the first database. In response to receiving the second interface performance data, the system determines the second attribute information corresponding to the second interface performance data; based on the second attribute information and the first attribute information, the system updates the statistical data corresponding to the interface performance indicators of the monitored object in real time.

4. The method according to claim 3, characterized in that, The step of updating the statistical data corresponding to the interface performance metrics of the monitored object based on the second attribute information and the first attribute information includes: Determine whether the second attribute information is the same as the first attribute information; If the second attribute information is the same as the first attribute information, based on the second interface performance data, update the first performance data sequence corresponding to the multiple interface performance indicators under the first attribute information; based on the updated first performance data sequence, update the first indicator statistical data corresponding to each of the interface performance indicators in the first database; If the second attribute information is different from the first attribute information, based on the second interface performance data and the second attribute information, a second performance data sequence corresponding to multiple interface performance indicators under the second attribute information is constructed in the first database; wherein, the second performance data sequence is associated with the second attribute information; For each interface performance metric under the second attribute information, based on the second performance data sequence corresponding to the interface performance metric, the second indicator statistical data corresponding to the interface performance metric under the second attribute information is determined in the first database.

5. The method according to claim 1, characterized in that, The second database has a pre-set data storage table; the data storage table contains multiple preset performance indicators and corresponding indicator columns; the indicator columns correspond to a designated storage area of ​​the second database; The step of storing the object performance data in a second database in response to receiving the object performance data includes: In response to receiving the first object performance data, determine the first object performance index corresponding to the first object performance data; The performance data corresponding to the first object performance index in the first object performance data is written into the first index column, so that the performance data corresponding to the first object performance index is stored in the specified storage area corresponding to the first index column; wherein, the first object performance index matches the preset performance index corresponding to the first index column.

6. The method according to claim 5, characterized in that, Before the step of writing the performance data corresponding to the first object performance index in the first object performance data into the first index column, the method further includes: Identify a second object performance indicator among the preset performance indicators that does not match the first object performance indicator; write the indicator column corresponding to the second object performance indicator into the specified data.

7. The method according to claim 5, characterized in that, The data storage table contains preset sorting rules; the sorting rules are used to indicate the sorting order of the object performance data in the data storage table; The step of writing the performance data corresponding to the first object performance index in the first object performance data into the first index column includes: Determine the object attribute information corresponding to the performance data of the first object; wherein, the object attribute information includes: the data owner information of the performance data of the first object, the acquisition task identifier of the performance data of the first object, the unit identifier of the monitoring unit of the monitored object, and the acquisition time of the performance data of the first object; the type of the monitoring unit includes: device type and process type; Based on the sorting rules and the object attribute information, the sorting order of the first object performance data in the data storage table is determined; According to the sorting order, the performance data corresponding to the first object performance index in the first object performance data is written into the first index column.

8. The method according to claim 7, characterized in that, The sorting rules include one or more of the following: The performance data of the objects corresponding to the same data owner information are arranged consecutively in the data storage table; The performance data of the objects corresponding to the same acquisition task identifier are arranged consecutively in the data storage table; The performance data of the objects corresponding to the monitoring units of the same type are arranged consecutively in the data storage table; The object performance data is sorted according to the order of the collection time.

9. The method according to claim 1, characterized in that, The step of obtaining statistical data on the target performance metric corresponding to the first object from the first database based on the target performance metric includes: When the target performance metric is an interface performance metric, retrieve the statistical data of the metric corresponding to the interface performance metric of the first object from the first database.

10. The method according to claim 1, characterized in that, The target performance metric includes the object performance metric of the first object; the data statistics request includes: a sorting request for the performance data corresponding to the object performance metric of the first object; The step of determining the target performance data corresponding to the target performance metric in the target storage area of ​​the second database based on the target performance metric, and determining the metric statistics corresponding to the target performance metric of the first object based on the performance data in the target storage area, includes: Determine the sorting method and the number of sorting operations from the sorting request; The performance data corresponding to the object performance index of the first object is determined to be in the target storage area of ​​the second database. The performance data corresponding to the object performance index of the first object is obtained from the target storage area. The performance data is sorted according to the sorting method. The performance data matching the sorting quantity is extracted from the sorting result and determined as the indicator statistics.

11. The method according to claim 1, characterized in that, The steps for collecting object performance data and interface performance data of the monitored object include: The system uses a pre-defined data acquisition component to collect object performance data and interface performance data of the monitored object. The collected object performance data and interface performance data are encapsulated into a message packet of a specified format and sent asynchronously to a preset backend service component; The step of storing the interface performance data in a first database in response to receiving the interface performance data includes: In response to the backend service component receiving the message packet, the message packet is parsed, and the parsed interface performance data is stored in the first database; The step of storing the object performance data in a second database in response to receiving the object performance data includes: In response to the backend service component receiving the message packet, the message packet is parsed, and the parsed object performance data is stored in the second database.

12. The method according to claim 1, characterized in that, Following the step of responding to a data acquisition task instruction for a monitored object, the method further includes: The metadata of the data acquisition task is collected and stored in a third database; wherein, the metadata includes: attribute information of the monitored object and configuration information corresponding to the data acquisition task.

13. The method according to claim 1, characterized in that, After the step of determining the metric statistics corresponding to the target performance metric of the first object from the target storage region, the method further includes: The configuration information corresponding to the data acquisition task is obtained from a pre-set third database, and the configuration information and the statistical data of the target performance indicators are determined as the response result of the data statistics request.

14. The method according to claim 1, characterized in that, The method further includes: In response to the completion of the data acquisition task, the interface performance data is stored in the second database; wherein, in the second database, the interface performance data is stored in different regions according to the interface performance indicators of the monitored object.

15. The method according to claim 1, characterized in that, Both the first database and the first database are deployed in a distributed cluster, which adopts a sharding and replication mechanism; The step of storing the interface performance data in the first database includes: The interface performance data is distributed to the first shard node in the first database cluster according to the preset sharding rules, and synchronously stored to the replica node corresponding to the first shard node. The step of storing object performance data in the second database includes: The object performance data is distributed to the second shard node in the second database cluster according to the preset sharding rules, and synchronously stored to the corresponding replica node of the second shard node.

16. The method according to claim 15, characterized in that, The method further includes: When a specified shard node fails, read and write requests for the specified shard node will be switched to the replica node corresponding to the specified shard node; wherein, the specified shard node includes: the first shard node and / or the second shard node.

17. A performance data storage device, characterized in that, The device includes: The first acquisition module is used to respond to the data acquisition task instruction for the monitored object and acquire the object performance data and interface performance data of the monitored object; The first update module is used to, in response to receiving the interface performance data, store the interface performance data in a first database, and update the statistical data of the interface performance indicators of the monitored object in the first database in real time based on the interface performance data. A first storage module is configured to, in response to receiving the object performance data, store the object performance data in a second database; wherein, in the second database, the object performance data is stored in regions according to the object performance indicators of the monitored object; The first determining module is used to respond to a data statistics request for a first object among the monitored objects, determine the target performance indicator to be statistically analyzed corresponding to the data statistics request, and based on the target performance indicator, obtain the indicator statistics data corresponding to the target performance indicator of the first object from the first database, or determine that the performance data corresponding to the target performance indicator is in the target storage area of ​​the second database, and determine the indicator statistics data corresponding to the target performance indicator of the first object based on the performance data in the target storage area.

18. An electronic device, characterized in that, The device includes a processor and a memory, the memory storing machine-executable instructions that can be executed by the processor, the processor executing the machine-executable instructions to implement the performance data storage method according to any one of claims 1-16.

19. A storage medium, characterized in that, The storage medium stores machine-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the performance data storage method according to any one of claims 1-16.

Images