Monitoring data collection method, device, electronic device and computer readable medium
By aggregating monitoring data based on request address and type information, the problem of high redundancy in monitoring data is solved, and storage costs and network bandwidth resources are saved.
Patent Information
- Application Number
- CN202011403365.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-12-04
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2040-12-04
Smart Images

Figure CN112307071B_ABST
Abstract
Description
Technical Field
[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a monitoring data acquisition method, device, electronic device, and computer-readable medium. Background Art
[0002] With the rapid development of computer technology, various client applications corresponding to the business platform have been developed one after another. In order to ensure the normal operation of the business platform, it is necessary to collect monitoring data from each client to monitor the operation status of the business platform. At present, when collecting monitoring data, the commonly used method is to send the collected monitoring data directly to the monitoring data storage device.
[0003] However, when the above method is used to collect monitoring data, the following technical problems often occur:
[0004] First, the collected monitoring data is not aggregated according to the request address information of the client request, resulting in high redundancy of the collected monitoring data, making the storage cost of monitoring data high, and thus resulting in a waste of network bandwidth resources;
[0005] Second, the collected monitoring data is not aggregated according to the type information of the monitoring data, which further leads to high redundancy of the collected monitoring data, making the cost of monitoring data storage high, and further resulting in a waste of network bandwidth resources. Summary of the invention
[0006] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.
[0007] Some embodiments of the present disclosure propose monitoring data collection methods, devices, electronic devices, and computer-readable media to solve one or more of the technical problems mentioned in the above background technology section.
[0008] In a first aspect, some embodiments of the present disclosure provide a monitoring data collection method, the method comprising: obtaining, from various clients supporting the collection of monitoring data, respective pre-aggregated monitoring data corresponding to respective client requests as pre-aggregated monitoring data sets; based on respective request address information of the above-mentioned respective client requests, performing aggregation processing on the above-mentioned pre-aggregated monitoring data sets to generate an aggregated monitoring data set; and sending the above-mentioned aggregated monitoring data set to a monitoring data storage device of a corresponding monitoring system.
[0009] In a second aspect, some embodiments of the present disclosure provide a monitoring data collection device, which includes: an acquisition unit, configured to acquire, from various clients supporting the collection of monitoring data, respective pre-aggregated monitoring data corresponding to respective client requests as pre-aggregated monitoring data sets; an aggregation processing unit, configured to perform aggregation processing on the above-mentioned pre-aggregated monitoring data sets based on respective request address information of the above-mentioned respective client requests to generate an aggregated monitoring data set; and a sending unit, configured to send the above-mentioned aggregated monitoring data set to a monitoring data storage device of a corresponding monitoring system.
[0010] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.
[0011] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.
[0012] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through the monitoring data collection method of some embodiments of the present disclosure, the waste of network bandwidth resources is reduced. Specifically, the inventors found that the reason for the waste of network bandwidth resources is that the collected monitoring data is not aggregated according to the request address information of the client request, resulting in a high redundancy of the collected monitoring data, which makes the cost of monitoring data storage high. Based on this, the monitoring data collection method of some embodiments of the present disclosure first obtains the various pre-aggregated monitoring data corresponding to each client request from each client that supports the collection of monitoring data as a pre-aggregated monitoring data set, and then, based on the various request address information of the above-mentioned each client request, the above-mentioned pre-aggregated monitoring data set is aggregated to generate an aggregated monitoring data set, thereby, the obtained pre-aggregated monitoring data can be aggregated according to the request address information of the client request, reducing the redundancy of the collected monitoring data. Thereby reducing the cost of monitoring data storage. And then reducing the waste of network bandwidth resources. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.
[0014] Figure 1is a schematic diagram of an application scenario of the monitoring data collection method according to some embodiments of the present disclosure;
[0015] Figure 2 is a flow chart of some embodiments of the monitoring data collection method according to the present disclosure;
[0016] Figure 3 are flow charts of other embodiments of the monitoring data collection method according to the present disclosure;
[0017] Figure 4 are flow charts of some further embodiments of the monitoring data collection method according to the present disclosure;
[0018] Figure 5 is a schematic diagram of the structure of some embodiments of the monitoring data acquisition device according to the present disclosure;
[0019] Figure 6 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION
[0020] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.
[0021] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.
[0022] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.
[0023] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".
[0024] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.
[0025] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.
[0026] Figure 1 It is a schematic diagram of an application scenario of the monitoring data collection method according to some embodiments of the present disclosure.
[0027] exist Figure 1 In the application scenario, first, the computing device 101 can obtain the pre-aggregated monitoring data corresponding to each client request from each client supporting the collection of monitoring data as the pre-aggregated monitoring data set 102. Then, the computing device 101 can aggregate the pre-aggregated monitoring data set 102 based on the request address information 103 of each client request to generate an aggregated monitoring data set 104. Finally, the computing device 101 can send the aggregated monitoring data set 104 to the monitoring data storage device 105 of the corresponding monitoring system.
[0028] It should be noted that the computing device 101 can be hardware or software. When the computing device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or as a single server or a single terminal device. When the computing device is embodied as software, it can be installed in the hardware devices listed above. It can be implemented as multiple software or software modules for providing distributed services, or as a single software or software module. No specific limitation is made here.
[0029] It should be understood that Figure 1 The number of computing devices in the embodiment is only illustrative. Any number of computing devices may be provided according to implementation requirements.
[0030] Continue to refer Figure 2 , shows a process 200 of some embodiments of the monitoring data collection method according to the present disclosure. The monitoring data collection method comprises the following steps:
[0031] Step 201 : obtaining, from each client supporting the collection of monitoring data, each pre-aggregated monitoring data corresponding to each client request as a pre-aggregated monitoring data set.
[0032] In some embodiments, the execution subject of the monitoring data collection method (for example Figure 1The computing device 101 shown in the figure can obtain the pre-aggregated monitoring data corresponding to the requests of each client from each client supporting the collection of monitoring data as a pre-aggregated monitoring data set through a wired connection method or a wireless connection method. Among them, the above-mentioned client can be an electronic device that sends a client request to the above-mentioned execution entity and is installed with an SDK (Software Development Kit) that supports the collection of monitoring data. A client installed with an SDK that supports the collection of monitoring data can collect monitoring data from JMX (Java Management Extensions). It should be pointed out that the above-mentioned wireless connection method may include but is not limited to 3G / 4G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.
[0033] The client request may include request address information, i.e., a URL. In practice, a URL is generally represented by a Uniform Resource Locator (URL). The pre-aggregated monitoring data may be monitoring data after the client aggregates the collected monitoring data within a predetermined period of time. The pre-aggregated monitoring data may include averaged monitoring data. The averaged monitoring data may be monitoring data in numerical form. For example, the way in which the client performs aggregation processing may include deduplication processing of the averaged monitoring data included in the collected monitoring data. For another example, the way in which the client performs aggregation processing may include averaging processing of the averaged monitoring data included in the collected monitoring data. The averaged monitoring data may include, but is not limited to, at least one of the following: response time (Queries Time, QT), query rate per second (Queries Per Second, QPS). The response time may be the time for the client to respond to the request for the client. The query rate per second may be the number of requests for the client processed by the client within a specified time. Thus, the monitoring data aggregated by each client may be used to be aggregated again by the execution subject.
[0034] As an example, the above-mentioned pre-aggregated monitoring data set can be {[QT: 2 seconds, QPS: 2], [QT: 3 seconds, QPS: 1], [QT: 1 second, QPS: 2]}. Among them, the pre-aggregated monitoring data [QT: 2 seconds, QPS: 2] can correspond to the request address information "https: / / ZZ.com / ZZ / Z" included in the client request corresponding to client "A". The pre-aggregated monitoring data [QT: 3 seconds, QPS: 1] can correspond to the request address information "https: / / ZZ.com / ZZ / Y" included in the client request corresponding to client "B". The pre-aggregated monitoring data [QT: 1 second, QPS: 2] can correspond to the request address information "https: / / ZZ.com / ZZ / Z" included in the client request corresponding to client "C".
[0035] Step 202: Aggregate the pre-aggregated monitoring data set based on the request address information of each client request to generate an aggregated monitoring data set.
[0036] In some embodiments, the execution subject may aggregate the pre-aggregated monitoring data set using the request address information of each client request to generate an aggregated monitoring data set. In practice, the execution subject may average the averaged monitoring data included in the pre-aggregated monitoring data with the same request address information in the pre-aggregated monitoring data set to generate aggregated monitoring data. Then, the pre-aggregated monitoring data that has not been averaged may be determined as aggregated monitoring data. Thus, an aggregated monitoring data set may be obtained. As an example, the pre-aggregated monitoring data with the same request address information in the pre-aggregated monitoring data set {[QT: 2 seconds, QPS: 2], [QT: 3 seconds, QPS: 1], [QT: 1 second, QPS: 2]} may be [QT: 2 seconds, QPS: 2] and [QT: 1 second, QPS: 2]. First, the execution subject may average [QT: 2 seconds, QPS: 2] and [QT: 1 second, QPS: 2] to generate aggregated monitoring data. Specifically, the averages of [QT: 2 seconds] and [QT: 1 second], [QPS: 2] and [QPS: 2] can be determined respectively to generate aggregated monitoring data [QT: 1.5 seconds, QPS: 2]. Then, the above-mentioned execution entity can determine the pre-aggregated monitoring data [QT: 3 seconds, QPS: 1] that has not been averaged as aggregated monitoring data. The obtained aggregated monitoring data set is {[QT: 1.5 seconds, QPS: 2], [QT: 3 seconds, QPS: 1]}. Therefore, the obtained pre-aggregated monitoring data can be aggregated according to the request address information requested by the client.
[0037] Step 203: Send the aggregated monitoring data set to the monitoring data storage device of the corresponding monitoring system.
[0038] In some embodiments, the execution subject may send the aggregated monitoring data set to the monitoring data storage device of the corresponding monitoring system. In practice, the execution subject may directly send the aggregated monitoring data set to the monitoring data storage device of the corresponding monitoring system. In practice, the execution subject may also first compress the aggregated monitoring data set to generate a compressed aggregated monitoring data set. For example, the LZW (Lenpel-Ziv&Welch) compression algorithm may be used to compress the aggregated monitoring data set. Then, the compressed aggregated monitoring data set may be sent to the monitoring data storage device of the corresponding monitoring system. The monitoring system may perform data analysis on the monitoring data to determine whether the monitoring data is an abnormal application or an electronic device on which the application is running. For example, the compressed aggregated monitoring data set may be sent to a server of the corresponding monitoring system for storing monitoring data. Thus, the redundancy of the monitoring data stored in the monitoring data storage device of the monitoring system is reduced.
[0039] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through the monitoring data collection method of some embodiments of the present disclosure, the waste of network bandwidth resources is reduced. Specifically, the inventors found that the reason for the waste of network bandwidth resources is that the collected monitoring data is not aggregated according to the request address information of the client request, resulting in a high redundancy of the collected monitoring data, which makes the cost of monitoring data storage high. Based on this, the monitoring data collection method of some embodiments of the present disclosure first obtains the various pre-aggregated monitoring data corresponding to each client request from each client that supports the collection of monitoring data as a pre-aggregated monitoring data set, and then, based on the various request address information of the above-mentioned each client request, the above-mentioned pre-aggregated monitoring data set is aggregated to generate an aggregated monitoring data set, thereby, the obtained pre-aggregated monitoring data can be aggregated according to the request address information of the client request, reducing the redundancy of the collected monitoring data. Thereby reducing the cost of monitoring data storage. And then reducing the waste of network bandwidth resources.
[0040] Continue to refer Figure 3 , shows a process 300 of another embodiment of the monitoring data collection method according to the present disclosure. The monitoring data collection method includes the following steps:
[0041] Step 301: Perform parameter configuration processing on each client supporting the collection of monitoring data.
[0042] In some embodiments, the execution subject of the monitoring data collection method (for example Figure 1 The computing device 101 shown in the figure can perform parameter configuration processing on each client supporting the collection of monitoring data through the following steps:
[0043] The first step is to obtain the configuration information corresponding to each of the above-mentioned clients to obtain a configuration information set. In practice, the above-mentioned execution subject can obtain the configuration information corresponding to each of the above-mentioned clients from the terminal through a wired connection method or a wireless connection method to obtain a configuration information set. Among them, the above-mentioned configuration information can be relevant information for configuring the collection method of collecting monitoring data, and can include but is not limited to at least one of the following: collection period, monitoring data storage path. For example, the above-mentioned configuration information set can be {[collection period: 10 minutes, monitoring data storage path: c: / XX / XX.txt], [collection period: 15 minutes, monitoring data storage path: c: / TT / TT.txt], [collection period: 10 minutes, monitoring data storage path: c: / SS / SS.txt]}.
[0044] The second step is to send each configuration information in the above configuration information set to the client corresponding to the above configuration information so that the above client can collect monitoring data according to the above configuration information. For example, the configuration information [collection period: 10 minutes, monitoring data storage path: c: / XX / XX.txt] can correspond to client "A". The configuration information [collection period: 15 minutes, monitoring data storage path: c: / TT / TT.txt] can correspond to client "B". The configuration information [collection period: 10 minutes, monitoring data storage path: c: / SS / SS.txt] can correspond to client "C". The above execution entity can send the above configuration information [collection period: 10 minutes, monitoring data storage path: c: / XX / XX.txt] to client "A" so that the above client "A" can collect monitoring data according to the above configuration information [collection period: 10 minutes, monitoring data storage path: c: / XX / XX.txt]. The above-mentioned execution subject may send the above-mentioned configuration information [collection period: 15 minutes, monitoring data storage path: c: / TT / TT.txt] to the client "B" so that the above-mentioned client "B" can collect monitoring data according to the above-mentioned configuration information [collection period: 15 minutes, monitoring data storage path: c: / TT / TT.txt]. The above-mentioned execution subject may send the above-mentioned configuration information [collection period: 10 minutes, monitoring data storage path: c: / SS / SS.txt] to the client "C" so that the above-mentioned client "C" can collect monitoring data according to the above-mentioned configuration information [collection period: 10 minutes, monitoring data storage path: c: / SS / SS.txt].
[0045] Through step 301, unified configuration of the configuration information related parameters of the above-mentioned various clients can be achieved, so that the above-mentioned various clients can collect monitoring data according to various configuration information.
[0046] Step 302 : obtaining, from each client supporting the collection of monitoring data, each pre-aggregated monitoring data corresponding to each client request as a pre-aggregated monitoring data set.
[0047] Step 303: Aggregate the pre-aggregated monitoring data set based on the request address information of each client request to generate an aggregated monitoring data set.
[0048] Step 304: Send the aggregated monitoring data set to the monitoring data storage device of the corresponding monitoring system.
[0049] In some embodiments, the specific implementation of steps 302-304 and the technical effects brought about can be referred to Figure 2 The corresponding steps 201-203 in the embodiments are not described in detail here.
[0050] from Figure 3 It can be seen that Figure 2 Compared with the description of some corresponding embodiments, Figure 3 The process 300 of the monitoring data collection method in some corresponding embodiments reflects the addition of a step of performing parameter configuration processing on each client supporting the collection of monitoring data. Therefore, the schemes described in these embodiments can realize the unified configuration of the configuration information related parameters of the above-mentioned various clients. Thus, the above-mentioned various clients can collect monitoring data according to various configuration information.
[0051] Further references Figure 4 , which shows a process 400 of some other embodiments of the monitoring data collection method. The process 400 of the monitoring data collection method includes the following steps:
[0052] Step 401 : obtaining, from each client supporting the collection of monitoring data, each pre-aggregated monitoring data corresponding to each client request as a pre-aggregated monitoring data set.
[0053] In some embodiments, the specific implementation of step 401 and the technical effects brought about can be referred to Figure 2 The corresponding step 201 in the embodiments will not be described in detail here.
[0054] Step 402: based on each request address information, group the pre-aggregated monitoring data set to generate a pre-aggregated monitoring data set.
[0055] In some embodiments, the execution subject of the monitoring data collection method (for example Figure 1 The computing device 101 shown can generate a pre-aggregated monitoring data set by following the steps below:
[0056] The first step is to deduplicate the above request address information and generate a deduplicated request address information set. For example, the above request address information can be {https: / / ZZ.com / ZZ / Z, https: / / ZZ.com / ZZ / Y, https: / / ZZ.com / ZZ / Z}. The first request address information and the third request address information are repeated, and one of the request address information can be randomly retained, and the generated deduplicated request address information set is {https: / / ZZ.com / ZZ / Z, https: / / ZZ.com / ZZ / Y}.
[0057] In the second step, at least one pre-aggregated monitoring data in the pre-aggregated monitoring data set corresponding to each de-duplication request address information in the de-duplication request address information set is determined as a pre-aggregated monitoring data group to obtain a pre-aggregated monitoring data group set. For example, the de-duplication request address information set may be {https: / / ZZ.com / ZZ / Z, https: / / ZZ.com / ZZ / Y}. The pre-aggregated monitoring data set may be {[QT: 2 seconds, QPS: 2], [QT: 3 seconds, QPS: 1], [QT: 1 second, QPS: 2]}. The execution entity may determine the two pre-aggregated monitoring data [QT: 2 seconds, QPS: 2] and [QT: 1 second, QPS: 2] corresponding to the de-duplication request address information [https: / / ZZ.com / ZZ / Z] in the de-duplication request address information set {https: / / ZZ.com / ZZ / Z, https: / / ZZ.com / ZZ / Y} as the pre-aggregated monitoring data group. The execution subject may determine the pre-aggregated monitoring data [QT: 3 seconds, QPS: 1] corresponding to the de-duplication request address information [https: / / ZZ.com / ZZ / Y] in the de-duplication request address information set {https: / / ZZ.com / ZZ / Z, https: / / ZZ.com / ZZ / Y} as the pre-aggregated monitoring data set. The obtained pre-aggregated monitoring data set is {[[QT: 2 seconds, QPS: 2], [QT: 1 second, QPS: 2]], [QT: 3 seconds, QPS: 1]}.
[0058] Through step 402, the processing object of the aggregation processing of the pre-aggregated monitoring data set can be refined.
[0059] Step 403 : Aggregate the pre-aggregated monitoring data in each pre-aggregated monitoring data group in the pre-aggregated monitoring data group set to generate aggregated monitoring data, thereby obtaining an aggregated monitoring data set.
[0060] In some embodiments, the pre-aggregated monitoring data may also include cumulative monitoring data. The cumulative monitoring data may be monitoring data in the form of quantity. The cumulative monitoring data may include, but is not limited to, the number of requests. The number of requests may be the cumulative number of URLs represented by the request address information corresponding to the client request. The way in which the client performs aggregation processing in step 201 may also include accumulating the cumulative monitoring data included in the collected monitoring data. In practice, first, the execution subject may average the averaged monitoring data included in each pre-aggregated monitoring data in the pre-aggregated monitoring data group to generate average monitoring data. Then, the execution subject may accumulate the cumulative monitoring data included in each pre-aggregated monitoring data in the pre-aggregated monitoring data group to generate accumulated monitoring data. Finally, the execution subject may generate aggregated monitoring data based on the average monitoring data and the accumulated monitoring data. In practice, the execution subject may combine the average monitoring data and the accumulated monitoring data to generate aggregated monitoring data.
[0061] As an example, the above-mentioned pre-aggregated monitoring data group may be [[QT: 2 seconds, QPS: 2, request volume: 3], [QT: 1 second, QPS: 2, request volume: 1]]. First, the above-mentioned execution entity may average the two averaged monitoring data [QT: 2 seconds, QPS: 2] and [QT: 1 second, QPS: 2] included in the two pre-aggregated monitoring data in the above-mentioned pre-aggregated monitoring data group to generate the average monitoring data [QT: 1.5 seconds, QPS: 2]. Then, the above-mentioned execution entity may accumulate the two cumulative monitoring data [request volume: 3] and [request volume: 1] included in the two pre-aggregated monitoring data in the above-mentioned pre-aggregated monitoring data group to generate the accumulated monitoring data [request volume: 4]. Finally, the above-mentioned execution entity may combine the above-mentioned average monitoring data [QT: 1.5 seconds, QPS: 2] and the above-mentioned accumulated monitoring data [request volume: 4] to generate the aggregated monitoring data [QT: 1.5 seconds, QPS: 2, request volume: 4].
[0062] As another example, the pre-aggregated monitoring data group may be [QT: 3 seconds, QPS: 1, request volume: 2]. First, the execution entity may average the averaged monitoring data [QT: 3 seconds, QPS: 1] included in the pre-aggregated monitoring data in the pre-aggregated monitoring data group to generate average monitoring data [QT: 3 seconds, QPS: 1]. Then, the execution entity may perform accumulation processing on the accumulated monitoring data [request volume: 2] included in the pre-aggregated monitoring data in the pre-aggregated monitoring data group to generate accumulated monitoring data [request volume: 2]. Finally, the execution entity may combine the average monitoring data [QT: 3 seconds, QPS: 1] and the accumulated monitoring data [request volume: 2] to generate aggregate monitoring data [QT: 3 seconds, QPS: 1, request volume: 2]. Thus, the aggregated monitoring data set is obtained as {[QT: 1.5 seconds, QPS: 2, request volume: 4], [QT: 3 seconds, QPS: 1, request volume: 2]}.
[0063] As an inventive point of an embodiment of the present disclosure, the above technical solution solves the second technical problem mentioned in the background technology, "the collected monitoring data is not aggregated according to the type information of the monitoring data, which further leads to a high redundancy of the collected monitoring data, making the cost of monitoring data storage high, and further leading to a waste of network bandwidth resources.". The factors that lead to the waste of network bandwidth resources are often as follows: the collected monitoring data is not aggregated according to the type information of the monitoring data, which further leads to a high redundancy of the collected monitoring data, making the cost of monitoring data storage high. If the above factors are solved, the effect of reducing the waste of network bandwidth resources can be achieved. In order to achieve this effect, the present disclosure aggregates the pre-aggregated monitoring data in each pre-aggregated monitoring data group obtained by grouping processing through the averaged monitoring data and the accumulated monitoring data included in the pre-aggregated monitoring data to generate aggregated monitoring data, and obtain an aggregated monitoring data set. Since the aggregation processing methods of averaging processing and accumulation processing are respectively adopted for the averaged monitoring data and the accumulated monitoring data, the collected monitoring data is aggregated according to the type information of the monitoring data. Thereby, the redundancy of the collected monitoring data is reduced, and the cost of monitoring data storage is reduced. As a result, the waste of network bandwidth resources is reduced.
[0064] Step 404: Send the aggregated monitoring data set to the monitoring data storage device of the corresponding monitoring system.
[0065] In some embodiments, the specific implementation of step 404 and the technical effects brought about can be referred to Figure 2 The corresponding step 203 in the embodiments will not be described in detail here.
[0066] from Figure 4It can be seen that Figure 2 Compared with the description of some corresponding embodiments, Figure 4 The process 400 of the monitoring data collection method in some corresponding embodiments embodies the steps of expanding the generation of aggregated monitoring data sets. Since the aggregation processing methods of averaging processing and accumulation processing are respectively adopted for the averaged monitoring data and the accumulated monitoring data, the collected monitoring data is aggregated according to the type information of the monitoring data. This reduces the redundancy of the collected monitoring data, thereby reducing the cost of monitoring data storage. As a result, the waste of network bandwidth resources is reduced.
[0067] Further references Figure 5 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a monitoring data acquisition device. These device embodiments are similar to Figure 2 Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices.
[0068] like Figure 5 As shown, the monitoring data collection device 500 of some embodiments includes: an acquisition unit 501, an aggregation processing unit 502 and a sending unit 503. The acquisition unit 501 is configured to acquire, from various clients supporting the collection of monitoring data, various pre-aggregated monitoring data corresponding to various client requests as pre-aggregated monitoring data sets; the aggregation processing unit 502 is configured to perform aggregation processing on the above-mentioned pre-aggregated monitoring data sets based on various request address information of the above-mentioned various client requests to generate an aggregated monitoring data set; the sending unit 503 is configured to send the above-mentioned aggregated monitoring data set to the monitoring data storage device of the corresponding monitoring system.
[0069] In an optional implementation of some embodiments, before the acquisition unit 501 of the monitoring data collection device 500, the monitoring data collection device 500 may further include: a parameter configuration processing unit configured to perform parameter configuration processing on each client supporting the collection of monitoring data.
[0070] In an optional implementation of some embodiments, the parameter configuration processing unit may be further configured to: obtain configuration information corresponding to each of the above-mentioned clients to obtain a configuration information set; and send each configuration information in the above-mentioned configuration information set to the client corresponding to the above-mentioned configuration information so that the above-mentioned client can collect monitoring data according to the above-mentioned configuration information.
[0071] In an optional implementation of some embodiments, the aggregation processing unit 502 of the monitoring data collection device 500 may be further configured to include: a group processing subunit and an aggregation processing subunit. The group processing subunit is configured to perform group processing on the pre-aggregated monitoring data set based on the above-mentioned individual request address information to generate a pre-aggregated monitoring data set. The aggregation processing subunit is configured to perform aggregation processing on each pre-aggregated monitoring data in each pre-aggregated monitoring data set in the pre-aggregated monitoring data set to generate aggregated monitoring data and obtain an aggregated monitoring data set.
[0072] In optional implementations of some embodiments, the above-mentioned group processing subunit can be further configured to: perform deduplication processing on the above-mentioned individual request address information to generate a deduplication request address information set; determine at least one pre-aggregate monitoring data in the above-mentioned pre-aggregate monitoring data set corresponding to each deduplication request address information in the above-mentioned deduplication request address information set as a pre-aggregate monitoring data group to obtain a pre-aggregate monitoring data group set.
[0073] In an optional implementation of some embodiments, the pre-aggregated monitoring data includes accumulated monitoring data.
[0074] In an optional implementation of some embodiments, the above-mentioned aggregation processing subunit can be further configured as: an accumulation processing subunit, configured to perform accumulation processing on each accumulated monitoring data included in the above-mentioned each pre-aggregation monitoring data to generate accumulated monitoring data.
[0075] In an optional implementation of some embodiments, the above-mentioned aggregation processing subunit may also include: a generation unit configured to generate aggregated monitoring data based on the above-mentioned accumulated monitoring data.
[0076] It is understood that the units described in the device 500 are similar to those described in the reference Figure 2 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the device 500 and the units included therein, and will not be described in detail here.
[0077] Reference below Figure 6 , which shows an electronic device (eg, Figure 1 A schematic diagram of the structure of the computing device 600 in FIG. Figure 6 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0078] like Figure 6As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.
[0079] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 6 Each block shown in the figure may represent one device, or may represent multiple devices as required.
[0080] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.
[0081] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
[0082] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.
[0083] The computer-readable medium may be included in the electronic device; or it may exist independently without being installed in the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains the pre-aggregated monitoring data corresponding to the requests of each client from each client supporting the collection of monitoring data as a pre-aggregated monitoring data set; aggregates the pre-aggregated monitoring data set based on the request address information of each client request to generate an aggregated monitoring data set; and sends the aggregated monitoring data set to the monitoring data storage device of the corresponding monitoring system.
[0084] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0085] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0086] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be arranged in a processor, for example, may be described as: a processor includes an acquisition unit, an aggregation processing unit, and a sending unit. Among them, the names of these units do not constitute a limitation on the unit itself under certain circumstances. For example, the acquisition unit may also be described as "a unit for obtaining each pre-aggregated monitoring data corresponding to each client request from each client supporting the collection of monitoring data as a pre-aggregated monitoring data set".
[0087] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.
[0088] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.
Claims
1. A monitoring data collection method, comprising: Obtaining, from each client supporting the collection of monitoring data, each pre-aggregated monitoring data corresponding to each client request as a pre-aggregated monitoring data set; Based on the request address information of each client request, the pre-aggregated monitoring data set is aggregated to generate an aggregated monitoring data set, wherein the aggregation includes accumulation and averaging of the pre-aggregated monitoring data with the same request address information; The aggregated monitoring data set is sent to a monitoring data storage device of a corresponding monitoring system.
2. The method according to claim 1, wherein: Before obtaining, from each client supporting the collection of monitoring data, each pre-aggregated monitoring data corresponding to each client request as a pre-aggregated monitoring data set, the method further includes: Perform parameter configuration processing on each client that supports collecting monitoring data.
3. The method according to claim 2, wherein: The parameter configuration processing for each client supporting the collection of monitoring data includes: Obtaining configuration information corresponding to each of the clients to obtain a configuration information set; Each piece of configuration information in the configuration information set is sent to a client corresponding to the configuration information so that the client can collect monitoring data according to the configuration information.
4. The method according to claim 1, wherein: The aggregating the pre-aggregated monitoring data set based on the request address information of each client request to generate an aggregated monitoring data set includes: Based on the respective request address information, grouping the pre-aggregated monitoring data set to generate a pre-aggregated monitoring data set; Aggregation processing is performed on each pre-aggregated monitoring data in each pre-aggregated monitoring data group in the pre-aggregated monitoring data group set to generate aggregated monitoring data, thereby obtaining an aggregated monitoring data set.
5. The method according to claim 4, wherein: The grouping of the pre-aggregated monitoring data set based on the respective request address information to generate a pre-aggregated monitoring data set includes: Deduplication processing is performed on each of the request address information to generate a deduplication request address information set; At least one pre-aggregated monitoring data in the pre-aggregated monitoring data set corresponding to each de-duplication request address information in the de-duplication request address information set is determined as a pre-aggregated monitoring data group to obtain a pre-aggregated monitoring data group set.
6. The method according to claim 4, wherein: The pre-aggregated monitoring data includes accumulated monitoring data; and The aggregating the individual pre-aggregated monitoring data in each pre-aggregated monitoring data group in the pre-aggregated monitoring data group set to generate aggregated monitoring data includes: Accumulation processing is performed on each accumulated monitoring data included in each pre-aggregated monitoring data to generate accumulated monitoring data.
7. The method according to claim 6, wherein: The aggregating the pre-aggregated monitoring data in each pre-aggregated monitoring data group in the pre-aggregated monitoring data group set to generate aggregated monitoring data further includes: Based on the accumulated monitoring data, aggregated monitoring data is generated.
8. A monitoring data acquisition device, comprising: An acquisition unit is configured to acquire, from each client supporting the collection of monitoring data, each pre-aggregated monitoring data corresponding to each client request as a pre-aggregated monitoring data set; an aggregation processing unit, configured to perform aggregation processing on the pre-aggregated monitoring data set based on the respective request address information of the respective client requests to generate an aggregated monitoring data set, wherein the aggregation processing includes accumulation processing and averaging processing of the pre-aggregated monitoring data with the same request address information; The sending unit is configured to send the aggregated monitoring data set to a monitoring data storage device of a corresponding monitoring system.
9. An electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.
10. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
