Operation parameter acquisition method and device, computer equipment, storage medium and computer program product

Abstract

The invention relates to an operation parameter acquisition method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: determining each target parameter, and dividing each target parameter into at least one parameter group according to an influence relationship among the target parameters; in any parameter group, the number of the target parameters which have the influence relationship with each other meets a preset number condition; and respectively carrying out parallel acquisition processing on each target parameter in each parameter group. By adopting the method, the parameter acquisition precision can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an operating parameter acquisition method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] When performing performance testing on computer equipment, it is often necessary to collect comprehensive operating parameters of a single or multiple machines, such as monitoring the system status of the machine and monitoring the application running status.

[0003] In order to improve the efficiency of parameter collection in the prior art, different operating parameters are often collected in parallel. However, when multiple parameters are read simultaneously, inaccurate parameter collection may occur due to the mutual influence between data sources or the mutual influence between the data collection tools used. Summary of the invention

[0004] Based on this, it is necessary to provide an operating parameter collection method, device, computer equipment, storage medium and computer program product to address the above technical problems.

[0005] In a first aspect, the present application provides a method for collecting operating parameters. The method comprises:

[0006] Determine each target parameter, and divide each target parameter into at least one parameter group according to the influence relationship between each target parameter; in any parameter group, the number of the target parameters having the influence relationship between each other satisfies a preset number condition;

[0007] The target parameters in each parameter group are collected and processed in parallel.

[0008] The above-mentioned operating parameter collection method divides each target parameter into multiple parameter groups according to the influence relationship between each target parameter when collecting the target parameters, and collects the target parameters in each parameter group in parallel. Therefore, when collecting each target parameter, the number of target parameters that will influence each other among the target parameters collected simultaneously and in parallel can be reduced, thereby improving the parameter collection accuracy.

[0009] In one embodiment, dividing each target parameter into at least one parameter group according to the influence relationship between each target parameter includes:

[0010] The target parallel collection quantity is estimated based on the data collection requirements of each target parameter;

[0011] According to the influence relationship between the target parameters and the target parallel collection quantity, the target parameters are divided into at least one parameter group, and the number of target parameters in any parameter group is less than or equal to the target parallel collection quantity.

[0012] In this embodiment, when collecting target parameters, the number of target parameters that can be collected simultaneously is estimated, and each target parameter is grouped according to the estimated number, which can avoid the situation where too many parameters are collected simultaneously, resulting in inaccurate parameters, and improve the accuracy of parameter collection.

[0013] In one embodiment, the influence relationship includes an actual influence relationship and a potential influence relationship, and the preset quantity condition includes that there are no target parameters that have the actual influence relationship with each other, and the number of target parameters that have the potential influence relationship with each other is less than a first preset threshold.

[0014] In this embodiment, the influence relationship is divided into actual influence relationship and potential influence relationship, and the preset quantity condition is set to that there are no target parameters with actual influence relationship between each other in the same parameter group, and the number of target parameters with potential influence relationship between each other is less than a first preset threshold. This can reduce the probability of influence between target parameters collected at the same time while taking into account the parameter collection efficiency.

[0015] In one embodiment, the method further comprises:

[0016] For any two of the target parameters, when there is an association relationship between the data sources of the two target parameters, it is determined that there is the potential influence relationship between the two target parameters.

[0017] In this embodiment, target parameters with correlation relationships between data sources are determined as target parameters with potential influence relationships between each other, and each target parameter is divided into different parameter groups according to preset quantity conditions. This can avoid the mutual influence of data sources of the target parameters resulting in inaccurate parameter collection, and can improve data collection accuracy.

[0018] In one embodiment, the method further comprises:

[0019] In the case where there is a target parameter group, performing multiple parallel acquisition processes and multiple serial acquisition processes on each of the target parameters in the target parameter group to obtain multiple parallel acquisition results and multiple serial acquisition results; the number of target parameters in the target parameter group that have the potential influence relationship with each other is greater than a second preset threshold;

[0020] When the difference between each of the parallel acquisition results and each of the serial acquisition results meets a preset condition, the target parameters in the target parameter group that have the potential influence relationship with each other are determined as target parameters that have the actual influence relationship with each other, and the target parameters in the target parameter group are re-divided into the parameter group.

[0021] In this embodiment, for a target parameter group in which there are many target parameters that have a potential influence relationship with each other, parallel collection and serial collection are respectively performed on the target parameters in the target parameter group, and based on the difference between the results obtained during parallel collection and serial collection, it is determined whether the potential influence relationship is an actual influence relationship. If the potential influence relationship is an actual influence relationship, the target parameters are re-divided into parameter groups. Therefore, the target parameters that have an actual influence relationship with each other can be determined in time during the parameter collection process, and the target parameter groups can be re-grouped. The accuracy of parameter collection can be improved.

[0022] In one of the embodiments, the data collection requirement includes a collection interval and a total collection duration, and the target parallel collection quantity is positively correlated with the collection interval and negatively correlated with the total collection duration.

[0023] In this embodiment, when collecting target parameters, the number of target parameters that can be collected simultaneously is estimated by the collection interval and the total collection time, and each target parameter is grouped according to the estimated number. This can avoid the situation where too many parameters are collected simultaneously, resulting in inaccurate parameters, and improve the parameter collection accuracy.

[0024] In a second aspect, the present application also provides an operating parameter acquisition device. The device comprises:

[0025] A first determination module is used to determine each target parameter, and divide each target parameter into at least one parameter group according to the influence relationship between each target parameter; in any parameter group, the number of the target parameters having the influence relationship between each other satisfies a preset number condition;

[0026] The first acquisition module is used to perform parallel acquisition processing on each of the target parameters in each of the parameter groups.

[0027] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, any one of the above methods is implemented.

[0028] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, any of the above methods is implemented.

[0029] In a fifth aspect, the present application further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, any of the above methods is implemented.

[0030] The above-mentioned operating parameter collection method, device, computer equipment, storage medium and computer program product, when collecting target parameters, divide each target parameter into multiple parameter groups according to the influence relationship between each target parameter, and collect the target parameters in each parameter group in parallel. Therefore, when collecting each target parameter, the number of target parameters that will affect each other among the target parameters collected simultaneously and in parallel can be reduced, thereby improving the parameter collection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A schematic diagram of a process of operating a parameter acquisition method in an embodiment;

[0032] Figure 2 is a flow chart of step 102 in one embodiment;

[0033] Figure 3 A schematic diagram of a process for verifying a potential impact relationship in an embodiment;

[0034] Figure 4 A schematic diagram of a process for collecting operating parameters using an integrated monitoring tool in one embodiment;

[0035] Figure 5 It is a structural block diagram of an operating parameter acquisition device in one embodiment;

[0036] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0038] In an exemplary embodiment, Figure 1 As shown, a method for collecting operating parameters is provided. This embodiment uses the method applied to a server as an example for illustration. It is understandable that the method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0039] Step 102, determine each target parameter, and divide each target parameter into at least one parameter group according to the influence relationship between each target parameter. In any parameter group, the number of target parameters that have an influence relationship with each other meets a preset number condition.

[0040] Among them, the target parameters are parameters that need to be collected when the computer equipment is subjected to performance testing. The existence of an influence relationship between multiple target parameters means that when multiple target parameters are collected in parallel, the multiple target parameters influence each other, resulting in that the values ​​of the collected target parameters may be inaccurate, or these multiple target parameters cannot be collected at the same time. The influence relationship between the target parameters can be obtained through preliminary experiments, or can be inferred based on the access restrictions of the computer equipment itself, the data source of each target parameter, and the collection tool used when collecting each target parameter. For example, when the computer equipment does not allow access to multiple network interfaces at the same time, it can be considered that there is an influence relationship between all target parameters related to multiple network interfaces (such as the comparison of sending bandwidth and receiving bandwidth of different network interfaces). For another example, when the collection tool needs to occupy a large amount of CPU resources when collecting certain target parameters, collecting CPU utilization while collecting these target parameters will cause inaccurate CPU utilization, so there is an influence relationship between these target parameters and CPU utilization.

[0041] In implementation, the user may specify the target parameters to be collected, or when the user does not specify the target parameters, the server may automatically select preset parameters as the target parameters. The server groups the target parameters according to the influence relationship between the target parameters, so that in each parameter group, the number of target parameters that have an influence relationship with each other meets the preset number condition.

[0042] The preset quantity condition is used to limit the maximum number of target parameters that have an impact relationship with each other in each parameter group. Since the preset quantity condition determines how many parameter groups each target parameter needs to be divided into, the maximum number specified by the preset quantity condition is negatively correlated with the acquisition accuracy of each target parameter, and positively correlated with the acquisition efficiency of each target parameter.

[0043] The preset number condition can be preset by those skilled in the art according to actual needs. For example, the maximum number can be reduced when the acquisition accuracy needs to be improved, and the maximum number can be increased when the acquisition efficiency needs to be improved.

[0044] The preset quantity condition can also be determined according to the target parameters to be collected. For example, the influence relationship can be graded. The higher the level, the greater the mutual influence between two target parameters with an influence relationship when they are collected in parallel. The maximum quantity can be negatively correlated with the highest influence level of each target parameter, that is, the higher the highest influence level, the smaller the maximum quantity.

[0045] The parameter groups can be divided by any constrained optimization algorithm. The optimization goal can be to minimize the number of target parameters that influence each other in the same parameter group, or to minimize the number of parameter groups. The constraints can be a preset quantity condition, a maximum number of parameter groups, or a minimum number of target parameters in each parameter group (which can be determined based on the time requirement for completing the parameter collection task).

[0046] Step 104 , performing parallel acquisition processing on each target parameter in each parameter group.

[0047] During implementation, the server collects each target parameter in a parameter group in parallel at a time. After sequentially collecting the target parameters in each parameter group, the server displays the value of the target parameter to the user, or generates a system log based on the target parameter and stores the system log. After the user retrieves the system log, the server displays the system log to the user.

[0048] The above-mentioned operating parameter collection method divides each target parameter into multiple parameter groups according to the influence relationship between each target parameter when collecting the target parameters, and collects the target parameters in each parameter group in parallel. Therefore, when collecting each target parameter, the number of target parameters that will influence each other among the target parameters collected simultaneously and in parallel can be reduced, thereby improving the parameter collection accuracy.

[0049] In an exemplary embodiment, Figure 2 As shown, the specific processing process of step 102 includes steps 202 to 204. Among them:

[0050] Step 202: estimate the target parallel collection quantity according to the data collection requirements of each target parameter.

[0051] The data collection requirements may include at least the total collection duration, collection frequency, and total collection quantity of the target parameters. The data collection requirements may be pre-specified by the user. If the user does not specify the data collection requirements, the preset requirements may be used as the data collection requirements. The target parallel collection quantity is the maximum number of target parameters that are collected in parallel at the same time.

[0052] In implementation, the server estimates the maximum number of target parameters to be collected in parallel based on the data collection requirements of each target parameter. The higher the collection frequency, the more data that needs to be collected for each target parameter, and the longer the total collection time, the more system resources will be occupied when collecting parameters, and it is more likely to cause the collected data to change compared to the data in the normal state of the system, resulting in inaccurate collected target parameters.

[0053] According to the data collection requirements of each target parameter, a parameter that can represent the total data collection requirements of each target parameter can be statistically calculated, and the target parallel collection quantity can be estimated based on this parameter. For example, the total collection duration, collection frequency and total collection quantity of each target parameter can be weighted and summed according to the weights of these three parameters, and the weighted summation results corresponding to each target parameter can be added together, and then the summed result can be divided by the total number of target parameters to obtain an average value, which can represent the total data collection requirements of each target parameter. The relationship between the average value and the optimal target parallel collection quantity can be obtained in advance through experiments, so that the target parallel collection quantity can be estimated by this average value in practical applications.

[0054] It is also possible to pre-estimate the system occupancy rate caused by collecting target parameters according to the data collection requirements of each target parameter, and determine a system occupancy rate threshold that will not affect the accuracy of the parameters. Sort each target parameter from large to small according to the system occupancy rate, and accumulate the system occupancy rate of each target parameter starting from the first position of the sorted sequence until the accumulated sum is greater than the system occupancy rate threshold. The number of accumulated target parameters at this time is used as the target parallel collection number.

[0055] Step 204 , according to the influence relationship between the target parameters and the target parallel collection quantity, the target parameters are divided into at least one parameter group, and the number of target parameters in any parameter group is less than or equal to the target parallel collection quantity.

[0056] During implementation, parameter groups are divided according to the influence relationship between each target parameter and the target parallel acquisition number, so that each parameter group not only meets the requirement that the number of target parameters that have an influence relationship with each other meets the preset number condition, but also meets the requirement that the number of target parameters is less than or equal to the target parallel acquisition number.

[0057] After adding the requirement that the number of target parameters is less than or equal to the target number of parallel acquisitions, this condition can be added to the constraints of the aforementioned embodiments, and then the parameter groups can be divided by any constrained optimization algorithm.

[0058] In this embodiment, when collecting target parameters, the number of target parameters that can be collected simultaneously is estimated, and each target parameter is grouped according to the estimated number, which can avoid the situation where too many parameters are collected simultaneously, resulting in inaccurate parameters, and improve the accuracy of parameter collection.

[0059] In an exemplary embodiment, the influence relationship includes an actual influence relationship and a potential influence relationship, and the preset quantity conditions include that there are no target parameters with actual influence relationships between each other, and the number of target parameters with potential influence relationships between each other is less than a first preset threshold.

[0060] Among them, the actual impact relationship refers to the verified simultaneous collection of the two target parameters will lead to inaccurate target parameters, or the two target parameters cannot be collected at the same time. The potential impact relationship refers to the fact that the simultaneous collection of the two target parameters may lead to inaccurate target parameters, but whether it will lead to inaccuracy has not yet been verified. The first preset threshold can be a preset value, or it can be a value determined in real time according to each target parameter. Among them, the method for determining the first preset threshold in real time according to each target parameter can refer to the method for determining the maximum number of representations of preset quantity conditions in real time according to each target parameter in the aforementioned embodiment, and the embodiments of the present application will not be repeated here.

[0061] In implementation, the preset number conditions can be set in the same parameter group, there are no target parameters that have an actual influence relationship with each other (that is, if there is an actual influence relationship between two target parameters, the two target parameters must belong to different parameter groups), and at the same time, the number of target parameters that have a potential influence relationship with each other is less than the first preset threshold, so that the accuracy of parameter collection is higher.

[0062] In this embodiment, the influence relationship is divided into actual influence relationship and potential influence relationship, and the preset quantity condition is set to that there are no target parameters in the same parameter group that have an actual influence relationship with each other, and the number of target parameters that have a potential influence relationship with each other is less than a first preset threshold. This can reduce the probability of influence between target parameters collected at the same time while taking into account the parameter collection efficiency.

[0063] In an exemplary embodiment, the method further includes:

[0064] For any two target parameters, when there is an association relationship between the data sources of the two target parameters, it is determined that there is a potential influence relationship between the two target parameters.

[0065] Among them, the existence of an association relationship between data sources means that the data sources of the two target parameters are the same data source, or there is a relationship such as dependency between the data sources of the two target parameters. When the data sources of the two target parameters are the same data source, multiple tasks accessing the same data source may cause inaccurate parameter reading, so it can be considered that there is a potential association relationship between such target parameters. For example, when simultaneously collecting parameters such as bandwidth, latency, and packet loss rate of a network port, due to the bandwidth upper limit of the network port, the collection tasks of each parameter may not be executed normally according to the collection frequency and collection quantity specified in the data collection requirements. For example, when reading data from the PMU (Performance Monitoring Unit) or PMC (Performance Monitoring Counter) built into the CPU at the same time, due to the unexpected setting of the relevant register bits, the collected target parameters may deviate from the normal value of the target parameters. Dependency refers to the dependence of obtaining the value of one target parameter on the value of another target parameter, such as CPU utilization is calculated based on user state time, so CPU utilization depends on user state time.

[0066] In implementation, if there is an association relationship between the data sources of two target parameters, it can be considered that the two target parameters may affect each other, that is, there is a potential influence relationship. The server can pre-record the association relationship between each data source. After obtaining the target parameters, the data source of each target parameter is determined, and each data source is matched with the association relationship between each data source recorded in advance. If there are target parameters with an association relationship between the data sources, it is recorded that there is a potential conflict relationship between the two target parameters, and then the number of target parameters that have an influence relationship in each parameter group can be limited according to the preset quantity conditions, and each target parameter can be allocated to each parameter group.

[0067] In this embodiment, target parameters with correlation relationships between data sources are determined as target parameters with potential influence relationships between each other, and each target parameter is divided into different parameter groups according to preset quantity conditions. This can avoid the mutual influence of data sources of the target parameters resulting in inaccurate parameter collection, and can improve data collection accuracy.

[0068] In an exemplary embodiment, Figure 3 As shown, the above method also includes:

[0069] Step 302: if there is a target parameter group, perform multiple parallel acquisition processes and multiple serial acquisition processes on each target parameter in the target parameter group to obtain multiple parallel acquisition results and multiple serial acquisition results; the number of target parameters in the target parameter group that have a potential impact relationship with each other is greater than a second preset threshold;

[0070] Step 304, when the difference between each parallel acquisition result and each serial acquisition result meets the preset condition, the target parameters in the target parameter group that have a potential influence relationship with each other are determined as target parameters that have an actual influence relationship with each other, and the target parameters in the target parameter group are re-divided into parameter groups.

[0071] Among them, the target parameter group refers to a parameter group in which the number of target parameters with potential influence relationships between each other is greater than the second preset threshold, and the second preset threshold can be set to a value slightly smaller than the first preset threshold. If a target parameter group exists, it means that when each target parameter in the target parameter group is collected in parallel, the probability of each target parameter being inaccurate is higher than that of other parameter groups. Therefore, it is possible to verify whether each target parameter in the target parameter group that has a potential influence relationship with each other has an actual influence relationship. If there is an actual influence relationship, it is necessary to re-divide the target parameters in the target parameter group into parameter groups.

[0072] In implementation, since the target parameters generally need to be collected multiple times, if there is a target parameter group, each target parameter in the target parameter group can be collected in parallel in these multiple collections, and each target parameter in the target parameter group can be collected serially. The embodiment of the present application does not specifically limit the number of specific parallel collections and the number of serial collections. For example, half of the total number of times the target parameters are collected can be serial collection, and the other half can be parallel collection. In order to make the determination of the actual impact relationship more accurate, the number of serial collections can also be made greater than the number of parallel collections, so that enough serial collection results can be obtained to compare with the parallel collection results. After parallel collection, a parallel collection result can be obtained, and the parallel collection result includes the value obtained for each target parameter collected this time. After serial collection, a serial collection result can be obtained, and the serial collection result also includes the value obtained for each target parameter collected this time.

[0073] The embodiments of the present application do not specifically limit the order of parallel acquisition and serial acquisition. In an exemplary embodiment, parallel acquisition and serial acquisition can be performed in a concentrated manner, that is, multiple serial acquisitions can be performed continuously after multiple parallel acquisitions are performed continuously, or multiple parallel acquisitions can be performed continuously after multiple serial acquisitions are performed continuously. This method can better compare the difference between the serial acquisition results and the serial acquisition results, and the difference between the parallel acquisition results and the parallel acquisition results. In another exemplary embodiment, parallel acquisition and serial acquisition can be performed alternately, that is, a parallel acquisition can be performed after a serial acquisition, and a parallel acquisition can be performed after a serial acquisition. Alternatively, a parallel acquisition can be performed after two serial acquisitions. This method can better compare the difference between the serial acquisition results and the parallel acquisition results.

[0074] After obtaining multiple parallel acquisition results and multiple serial acquisition results, the differences between the multiple serial acquisition results and the multiple parallel acquisition results can be compared, and it can be determined whether the differences meet the preset conditions. The differences between the multiple serial acquisition results and the multiple parallel acquisition results may include, but are not limited to, the difference between each acquisition result and the previous acquisition result (regardless of whether the previous one is a parallel acquisition result or a serial acquisition result), the difference between each acquisition result and the previous acquisition result with the same acquisition mode (acquisition mode refers to serial or parallel), the difference between each acquisition result and the previous result with a different acquisition mode, etc. For any two acquisition results, the difference between the two acquisition results can be obtained by subtracting the values ​​obtained from the acquisition of the same target parameter in the two acquisition results.

[0075] If parallel acquisition has no effect on the accuracy of the parameters, that is, if there is no actual influence relationship between the target parameters in the target parameter group, then the difference between the serial acquisition results and the serial acquisition results, the difference between the parallel acquisition results and the parallel acquisition results, and the difference between the serial acquisition results and the parallel acquisition results should be small. Therefore, the preset conditions can be set accordingly according to this principle.

[0076] In order to eliminate the influence of natural changes in parameter values ​​on the determination of the actual influence relationship, when parallel acquisition and serial acquisition are performed in a centralized manner, the first difference between each two parallel acquisition results with adjacent acquisition sequences can be determined, and the second difference between each two serial acquisition results with adjacent acquisition sequences can be determined, and the first difference characterization parameter and the second difference characterization parameter are obtained by statistics for the first difference and the second difference, respectively. The first difference characterization parameter may include but is not limited to the average value of the first difference, the variance of the first difference, the maximum value of the first difference, etc. Similarly, the second difference characterization parameter may also include the average value, variance, maximum value, etc. of the second difference.

[0077] The preset condition can be set as the difference between the first difference characterization parameter and the second difference characterization parameter is less than the preset threshold. The difference between the first difference characterization parameter and the second difference characterization parameter refers to the difference of the same type of parameters in the first difference characterization parameter and the second difference characterization parameter, such as the difference between the average value of the first difference and the average value of the second difference, and the difference between the variance of the first difference and the variance of the second difference. The preset threshold value may include a fixed value set separately for the difference of each type of parameter, such as a threshold for the difference of the average value, a threshold for the difference of the variance, etc. When any difference in the difference of all types of parameters is greater than the threshold corresponding to the type, it is determined that the difference between each parallel acquisition result and each serial acquisition result meets the preset condition. Or the preset threshold value can also be determined in real time according to the value of the first difference characterization parameter and the second difference characterization parameter, such as for each type of parameter, taking the larger value of the parameter of the type in the first difference characterization parameter and the second difference characterization parameter, and then multiplying the larger value by a preset ratio value to obtain the preset threshold value of the parameter of the type.

[0078] When parallel acquisition and serial acquisition are performed crosswise, a first difference between every two parallel acquisition results with adjacent acquisition sequences can be determined, a second difference between every two serial acquisition results with adjacent acquisition sequences can be determined, and a third difference between every two serial acquisition results and parallel acquisition results with adjacent acquisition sequences can be determined. For example, when the crosswise acquisition sequence is parallel, serial, parallel, parallel, serial, serial, a first difference can be determined for the third and fourth acquisition results, three third differences can be determined for the first and second, second and third, fourth and fifth acquisition results, and a second difference can be determined for the fifth and sixth acquisition results.

[0079] Then, the first difference characterization parameter, the second difference characterization parameter, and the third difference characterization parameter can be calculated by referring to the method of calculating the first difference characterization parameter and the second difference characterization parameter during the aforementioned centralized collection. The preset condition can be set to that the difference between the first, second, and third difference characterization parameters is greater than the preset threshold, or to that at least one of the differences between the two is greater than the preset threshold, which is not specifically limited in the embodiments of the present application.

[0080] If the difference between each parallel acquisition result and each serial acquisition result meets the preset condition, it indicates that parallel acquisition will have a certain impact on the accuracy of the parameter. The target parameters in the target parameter group that have a potential impact relationship with each other can be determined as target parameters that have an actual impact relationship with each other, and the target parameters in the target parameter group can be re-divided into parameter groups according to the preset quantity condition.

[0081] In this embodiment, for a target parameter group in which there are many target parameters that have a potential influence relationship with each other, parallel collection and serial collection are respectively performed on the target parameters in the target parameter group, and based on the difference between the results obtained during parallel collection and serial collection, it is determined whether the potential influence relationship is an actual influence relationship. If the potential influence relationship is an actual influence relationship, the target parameters are re-divided into parameter groups. Therefore, the target parameters that have an actual influence relationship with each other can be determined in time during the parameter collection process, and the target parameter groups can be re-grouped. The accuracy of parameter collection can be improved.

[0082] In an exemplary embodiment, the data collection requirements include a collection interval and a total collection duration, and the target parallel collection quantity is positively correlated with the collection interval and negatively correlated with the total collection duration.

[0083] The collection interval refers to the time interval between two consecutive collections of a target parameter, and the total collection duration refers to the total time length for collecting a target parameter.

[0084] In implementation, since the shorter the collection interval and the longer the total collection time, the greater the system load of the server for parallel collection, the target parallel collection quantity can be positively correlated with the collection interval and negatively correlated with the total collection time.

[0085] When the data collection requirements for each target parameter are different, the shortest collection interval among the data collection requirements of each target parameter can be used as the target collection interval, and the longest total collection time among the data collection requirements of each target parameter can be used as the target collection total time, and the target parallel collection quantity is positively correlated with the target collection interval and negatively correlated with the target collection total time.

[0086] In this embodiment, when collecting target parameters, the number of target parameters that can be collected simultaneously is estimated by the collection interval and the total collection time, and each target parameter is grouped according to the estimated number. This can avoid the situation where too many parameters are collected simultaneously, resulting in inaccurate parameters, and improve the parameter collection accuracy.

[0087] In an exemplary embodiment, the method further includes:

[0088] In response to the termination signal, the parallel collection threads corresponding to the target parameters are terminated, and the parameter collection process is ended.

[0089] The termination signal is a signal manually sent by the user, which is used to indicate that the running parameter collection process needs to be terminated. Each target parameter corresponds to a parallel collection thread.

[0090] During implementation, if the user manually sends a termination signal (SIGINT signal in LINUX system) during the running of the above-mentioned parameter collection process, the server will synchronously destroy the parallel collection threads for collecting various target parameters to avoid the situation where the parallel collection threads have not yet ended after the main process exits, resulting in the formation of zombie processes in the background.

[0091] In an exemplary embodiment, if the operating parameters are collected for multiple computer devices at the same time, the method further includes:

[0092] Obtain the IP address and login information of each computer device;

[0093] The IP addresses of each computer device are accessed sequentially, and the currently accessed computer device is logged in based on the login information, and each target parameter is determined, and each target parameter is divided into at least one parameter group according to the influence relationship between the target parameters, and each target parameter in each parameter group is collected and processed in parallel.

[0094] Among them, the IP address and login information of each computer device can be recorded in a temporary file. When executing the operation parameter collection method, the server can automatically identify and check the format of the temporary file, and sequentially access and collect operation parameters for each computer device. Before logging into the computer device, the computer device can be ping checked and ssh checked. If the login fails, the computer device is skipped and the operation parameter collection of the next computer device is continued. When logging into the computer device, the expect statement block in the LINUX system can be used, and the spawn, expect, and send command combination can be used to automatically log in to the remote host.

[0095] It should be noted that if target parameter collection is performed for only one computer device, the step of verifying the integrity of each target parameter can be performed after each target parameter is collected. If target parameter collection is performed for multiple computer devices, the step of verifying the integrity can be skipped after completing the target parameter collection for each computer device, so as to avoid the situation of quitting midway during the process of continuously collecting the operating parameters of multiple computer devices.

[0096] In an exemplary embodiment, the method further includes:

[0097] According to the data source corresponding to each target parameter, the log generation format is determined, and the operation parameter log is generated according to the log generation format.

[0098] Among them, the log generation format may at least include the parallel display of the same target parameter of multiple identical components, the parallel display of all target parameters of the same component, and the parallel display of all target parameters of the computer device. The log generation format defaults to the parallel display of all target parameters of the computer device. If there are multiple target parameters corresponding to the same data source, it can be determined that the log generation format also includes the parallel display of all target parameters of the same component. If there are multiple target parameters corresponding to the same category of data sources, it is determined that the log generation format also includes the parallel display of the same target parameter of multiple identical components. Among them, when the hardware types corresponding to two data sources are the same, the data source categories are the same, for example, the data source categories corresponding to different network cards are the same, and the data source categories corresponding to different disks are the same. An operation parameter log can be generated respectively according to each format contained in the log generation format.

[0099] In an exemplary embodiment, the aforementioned embodiment may be implemented by an integrated monitoring tool. The integrated monitoring tool may integrate various basic monitoring tools in the LINUX system and be written in bash language to achieve compatibility with the LINUX system. The integrated monitoring tool may include a static parameter collection module, a running system parameter collection module, a running application parameter collection module, a log packaging module and a remote multi-host information collection module. When the integrated monitoring tool is running on a single computer device, the static parameter collection module, the running system parameter collection module, the running application parameter collection module and the log packaging module are sequentially executed. When running on multiple computer devices, the remote multi-host information collection module is executed, and the first four modules are sequentially executed on each computer device. When executing the first three modules to collect parameters, the aforementioned embodiments may be sequentially executed to determine each target parameter, divide each target parameter into at least one parameter group according to the influence relationship between each target parameter, and perform parallel collection and processing on each target parameter in each parameter group. See. Figure 4 The figure shows a schematic diagram of various steps performed when collecting parameters for a single computer device and multiple computer devices respectively.

[0100] The static parameter collection module is used to collect static information of the hardware and software of computer equipment, including CPU information (model, manufacturer, series, quantity, instruction set, cache, characteristics, frequency, NUMA (Non-Uniform Memory Access), power consumption, etc.), memory information (model, manufacturer, specification parameters, quantity, status, functional characteristics, etc.), disk information (hardware information, block device and driver parameters, file system and related attributes, etc.), network card information (hardware information, driver and protocol stack related characteristics and parameters), RAID (Redundant Arrays of Independent Disks) card information (hardware information, driver, firmware, RAID parameters, etc.), system information (operating system version, kernel version, kernel startup parameters and runtime parameters, glibc (GNU C Library, GNU C runtime library) version, system time, compiler version, etc.), server basic information (FRU (Field Replaceable Unit, field replaceable unit) information, sensor information, power consumption information, BIOS (Basic Input / Output System, basic input / output system) version, BMC (Baseboard Management System) version, etc.). Controller, Baseboard Management Controller) firmware version, motherboard information, serial number, etc.). You can write an "execute_and_log" function interface for this module. When calling this function, you can execute the functions of querying command records, outputting records, and writing log files, which is convenient for subsequent review.

[0101] The log files generated in this section can be placed in a directory in the baseinfo-$date-$time format to automatically distinguish the results of multiple collections.

[0102] The operating system parameter collection module is used to perform dynamic analysis of the whole machine status when the business is running. Users can specify data collection requirements for this module. The data collection requirements may include two inputs: total collection time - t and sampling interval - i. If no corresponding parameter is input, the default total collection time of 30 seconds and the interval of 1 second are used. The parameters that can be collected by this module include: the memory usage (heap, stack, private memory, etc.) of each component of each NUMA node of the whole system, the CPU time usage of user state, kernel state and other types of processes and the memory capacity usage of shared memory, logical memory and other types, the frequency and utilization status of each CPU core of the server, all systemd services running in the system, the sending and receiving bandwidth and number of packets of all network cards and network ports, the read and write IOPS (Input / Output Operations Per Second) of all block devices, bandwidth, latency and other information, various CPU interrupt statistics including PCIe device interrupts, system overall perf and perfsched analysis, the real-time memory bandwidth of cacheable memory and non-cacheable memory of each memory channel of the CPU, and the memory access bandwidth of multi-NUMA architecture CPUs between NUMA nodes, CPU utilization, NUMA location display and statistics of CPU usage at the thread level of the top N processes in memory utilization (N can be customized), and NUMA topology display and statistics of memory usage at the dynamic library level. It plays an important role in the performance diagnosis of multi-NUMA architecture servers.

[0103] The log files generated in this section are placed in a directory in the "runlog-$date-$time" format to automatically distinguish the results of multiple collections.

[0104] The running application parameter collection module is used to perform performance analysis on a single process. This module contains two running modes: attaching an existing running program through PID and starting a process from this module and monitoring the entire life cycle of this process. This module can accept three parameter inputs: process PID -p, total collection time -t, and sampling interval -i. If there is a -p parameter, the running mode of attaching an existing running program through PID is adopted; if there is no -p parameter, the running mode of starting a process from this module and monitoring the entire life cycle of this process is adopted. If the last two parameters are not input, the script default total collection time of 30 seconds and interval of 1 second are adopted.

[0105] The parameters that can be collected by this module are: CPU, memory, network, disk, etc. of the process, which are similar to the parameters that can be collected by the previous module. This module can automatically print the complete test command according to the PID of the monitored process, and distinguish the output in yellow. The various log files generated in this part are uniformly placed in the directory of the "runlog-process-$command_keyword-$time_stamp" format, so as to automatically distinguish the results of multiple collections of different processes.

[0106] The log automatic packaging module is used to automatically package the logs generated by the first three modules into a compressed package in the format of "logs-$host_serial-$host_ip-$time_stamp.tar.gz", where logs-$host represents the identifier of the computer device whose parameters are collected, serial-$host_ip represents the IP address of the computer device whose parameters are collected, and $time_stamp represents the timestamp, which makes it easier for technicians to distinguish logs from different computer devices. This module uses tar packaging and gzip format compression, which is a packaging and compression tool that comes with the OS, and no additional tools need to be installed.

[0107] The remote multi-host information collection module is used to collect batch information from multiple hosts at the same time. This module can first record the IP and login password information of the computer device to be collected in a temporary file. During operation, this module automatically identifies and checks the format of this file, and starts to collect the operating parameters of the host corresponding to the IP in the temporary file in parallel. Before collecting the operating parameters, the code segments contained in the above four modules that may cause midway exit (for example, checking whether a certain indicator collection tool is complete and not damaged, whether the parameter format of a certain indicator collection tool meets the requirements, checking whether the user has sufficient permissions, etc.) are deleted to collect as much remote host information as possible without interruption, and the error information and interrupt module redirection records can be recorded in a file, and sent back to the server with the integrated monitoring tool installed together with the log information, so as to continue the last collection.

[0108] When starting to collect running parameters, ping and ssh checks are performed on each remote host. If the login fails, the host will be prompted and skipped, and other host collections will continue. Then, the expect statement block can be used to automatically log in to the remote host using the spawn, expect, and send command combination, and the first four modules can be automatically executed. The packaged files sent back are named logs-$machine serial number-$IP-$timestamp and are placed in the logs-$IP directory. You can also set the timeout seconds for each connection according to the execution time of the script.

[0109] This module packages the logs into a compressed package with the name format of "logs-remote-from-$host_serial-$host_ip-$time_stamp.tar.gz", which can be distinguished from the local log naming.

[0110] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0111] Based on the same inventive concept, the embodiment of the present application also provides an operation parameter collection device for implementing the operation parameter collection method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more operation parameter collection device embodiments provided below can refer to the limitations of the operation parameter collection method above, and will not be repeated here.

[0112] In one embodiment, Figure 5 As shown, an operating parameter acquisition device 500 is provided, comprising: a first determination module 502, and an acquisition module 504, wherein:

[0113] A first determination module 502 is used to determine each target parameter, and divide each target parameter into at least one parameter group according to the influence relationship between each target parameter; in any parameter group, the number of the target parameters having the influence relationship between each other satisfies a preset number condition;

[0114] The first acquisition module 504 is used to perform parallel acquisition processing on each of the target parameters in each of the parameter groups.

[0115] In an exemplary embodiment, the first determining module 504 is further configured to:

[0116] The target parallel collection quantity is estimated based on the data collection requirements of each target parameter;

[0117] According to the influence relationship between the target parameters and the target parallel collection quantity, the target parameters are divided into at least one parameter group, and the number of target parameters in any parameter group is less than or equal to the target parallel collection quantity.

[0118] In an exemplary embodiment, the influence relationship includes an actual influence relationship and a potential influence relationship, and the preset quantity condition includes that there are no target parameters that have the actual influence relationship with each other, and the number of target parameters that have the potential influence relationship with each other is less than a first preset threshold.

[0119] In an exemplary embodiment, the apparatus further comprises:

[0120] The second determination module is used to determine, for any two of the target parameters, that there is a correlation relationship between the data sources of the two target parameters, whether the potential influence relationship exists between the two target parameters.

[0121] In an exemplary embodiment, the apparatus further comprises:

[0122] A second acquisition module is used for, when there is a target parameter group, performing multiple parallel acquisition processes and multiple serial acquisition processes on each of the target parameters in the target parameter group to obtain multiple parallel acquisition results and multiple serial acquisition results; the number of target parameters in the target parameter group that have the potential influence relationship with each other is greater than a second preset threshold;

[0123] The third determination module is used to determine the target parameters in the target parameter group that have the potential influence relationship with each other as target parameters that have the actual influence relationship with each other, and re-divide the target parameters in the target parameter group into the parameter group when the difference between each of the parallel acquisition results and each of the serial acquisition results meets the preset conditions.

[0124] In an exemplary embodiment, the data collection requirement includes a collection interval and a total collection duration, and the target parallel collection quantity is positively correlated with the collection interval and negatively correlated with the total collection duration.

[0125] Each module in the above device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module above.

[0126] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 6As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for collecting operating parameters is implemented.

[0127] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0128] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0129] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0130] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0131] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0132] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0133] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0134] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for collecting operating parameters, characterized in that: The method comprises: Determine each target parameter, and divide each target parameter into at least one parameter group according to the influence relationship between each target parameter; in any parameter group, the number of the target parameters having the influence relationship between each other satisfies a preset number condition; The target parameters in each parameter group are collected and processed in parallel.

2. The method according to claim 1, characterized in that The dividing the target parameters into at least one parameter group according to the influence relationship between the target parameters includes: The target parallel collection quantity is estimated based on the data collection requirements of each target parameter; According to the influence relationship between the target parameters and the target parallel collection quantity, the target parameters are divided into at least one parameter group, and the number of target parameters in any parameter group is less than or equal to the target parallel collection quantity.

3. The method according to claim 1, characterized in that The influence relationship includes an actual influence relationship and a potential influence relationship, and the preset quantity condition includes that there are no target parameters that have the actual influence relationship with each other, and the number of target parameters that have the potential influence relationship with each other is less than a first preset threshold.

4. The method according to claim 3, characterized in that The method further comprises: For any two of the target parameters, when there is an association relationship between the data sources of the two target parameters, it is determined that there is the potential influence relationship between the two target parameters.

5. The method according to claim 3, characterized in that: The method further comprises: In the case where there is a target parameter group, performing multiple parallel acquisition processes and multiple serial acquisition processes on each of the target parameters in the target parameter group to obtain multiple parallel acquisition results and multiple serial acquisition results; the number of target parameters in the target parameter group that have the potential influence relationship with each other is greater than a second preset threshold; When the difference between each of the parallel acquisition results and each of the serial acquisition results meets a preset condition, the target parameters in the target parameter group that have the potential influence relationship with each other are determined as target parameters that have the actual influence relationship with each other, and the target parameters in the target parameter group are re-divided into the parameter group.

6. The method according to claim 1, characterized in that The data collection requirement includes a collection interval and a total collection duration. The target parallel collection quantity is positively correlated with the collection interval and negatively correlated with the total collection duration.

7. An operating parameter collection device, characterized in that: The device comprises: A first determination module is used to determine each target parameter, and divide each target parameter into at least one parameter group according to the influence relationship between each target parameter; in any parameter group, the number of the target parameters having the influence relationship between each other satisfies a preset number condition; The first acquisition module is used to perform parallel acquisition processing on each of the target parameters in each of the parameter groups.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images