Methods and devices for determining log current limit values, storage media and electronic devices

CN117591346BActive Publication Date: 2026-11-17QINGDAO HAIER TECH +2
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202311505875.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-11-10
Publication Date
2026-11-17
Estimated Expiration
2043-11-10

AI Technical Summary

Technical Problem

[0005]本申请实施例提供了一种日志限流值的确定方法及装置、存储介质及电子装置,以至少解决相关技术中无法动态限制输入到程序中进行处理的日志数量的问题

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117591346B_ABST
    Figure CN117591346B_ABST
Patent Text Reader

Abstract

This application discloses a method, apparatus, storage medium, and electronic device for determining a log rate limit value, relating to the field of smart home technology. The method for determining the log rate limit value includes: determining the log redundancy of a target program within a first time period and the memory size used by the target program to process logs within the first time period, wherein the log redundancy is the amount of unprocessed logs by the target program; determining a log rate limit value for the target program in a second time period based on the log redundancy and the memory size, wherein the log rate limit value is the maximum number of logs allowed to be processed by the target program, wherein the first time period precedes the second time period, and the first and second time periods have the same duration. This solves the problem in related technologies where the number of logs input into a program for processing cannot be dynamically limited.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of smart home technology, and more specifically, to a method and apparatus for determining log current limit values, a storage medium, and an electronic device. Background Technology

[0002] As the program iterates, critical logs become increasingly important for problem-solving, leading to a surge in their generation. However, the processing and memory capabilities of the related systems are limited. When the processing volume exceeds capacity, queues typically accumulate, causing response times to increase rapidly and resulting in system crashes. Conversely, unprocessed log data buildup increases memory usage, which, when it reaches a certain level, negatively impacts the CPU's processing power. Without proper control, this can easily cause the entire related system to crash. Therefore, to ensure the system can operate normally under sudden traffic surges, dynamic rate limiting of log input to the program is necessary. This allows the system to provide maximum processing capacity while remaining robust and preventing it from being overwhelmed.

[0003] There is no effective solution yet for the problem that the number of logs input into the program for processing cannot be dynamically limited in related technologies.

[0004] Therefore, it is necessary to improve the relevant technology to overcome the aforementioned defects. Summary of the Invention

[0005] This application provides a method and apparatus for determining log flow limit values, a storage medium, and an electronic device, to at least solve the problem in related technologies that the number of logs input into a program for processing cannot be dynamically limited.

[0006] According to one aspect of the embodiments of this application, a method for determining a log rate limit value is provided, comprising: determining the log redundancy amount of a target program in a first time period, and the memory size used by the target program to process logs in the first time period, wherein the log redundancy amount is the amount of logs that the target program has not processed; determining a log rate limit value of the target program in a second time period based on the log redundancy amount and the memory size, wherein the log rate limit value is the maximum number of logs allowed to be processed from the logs generated by the target program.

[0007] In an exemplary embodiment, determining the log redundancy of a target program within a first time period includes: determining the log input volume and log processing volume of the target program within the first time period, wherein the log input volume is the number of logs generated by the target program, and the log processing volume is the number of logs actually processed by the target program; and determining the difference between the log input volume and the log processing volume as the log redundancy.

[0008] In an exemplary embodiment, determining the memory size used by the target program to process logs during the first time period includes: obtaining the available memory size of the target program during the first time period, and a first weight corresponding to the available memory size; determining the product of the available memory size and the first weight; determining the estimated memory size for the target program to process the log redundancy; and determining the ratio of the available memory size to the product as the memory size used by the target program to process logs during the first time period.

[0009] In an exemplary embodiment, determining the memory size used by the target program to process logs during the first time period includes: obtaining the processor utilization rate of the target program during the first time period; determining the available memory ratio of the target program during the first time period, wherein the available memory ratio is the ratio of the available memory size of the target program to the total available memory; and determining the memory size used by the target program to process logs during the first time period based on the utilization rate and the available memory ratio.

[0010] In an exemplary embodiment, determining the memory size used by the target program to process logs during the first time period based on the utilization rate and the available memory ratio includes: determining a second weight corresponding to the utilization rate; determining the used memory ratio based on the available memory ratio, and determining a third weight corresponding to the used memory ratio; and determining the sum of the product of the utilization rate and the second weight and the product of the used memory ratio and the third weight as the memory size used by the target program to process logs during the first time period.

[0011] In an exemplary embodiment, determining the log rate limiting value for the target program in a second time period based on the log redundancy and the memory size includes: determining a first threshold corresponding to the memory size, wherein the first threshold is a memory backlog threshold for the target program; determining the log rate limiting value for the first time period; if the memory size is greater than or equal to the first threshold, determining a first adjustment ratio corresponding to the log rate limiting value for the first time period based on the memory size; and determining the log rate limiting value for the second time period based on the log rate limiting value for the first time period and the first adjustment ratio.

[0012] In an exemplary embodiment, after determining the log rate limit value for the first time period, the method further includes: if the memory size is less than the first threshold, determining a second adjustment ratio corresponding to the log redundancy based on the memory size; and determining the log rate limit value for the second time period based on the log rate limit value for the first time period, the log redundancy, and the second adjustment ratio.

[0013] According to another aspect of the embodiments of this application, a device for determining a log rate limit value is also provided, comprising: a first determining module, configured to determine the log redundancy amount of a target program in a first time period and the memory size used by the target program to process logs in the first time period, wherein the log redundancy amount is the amount of logs that the target program has not processed; and a second determining module, configured to determine a log rate limit value of the target program in a second time period based on the log redundancy amount and the memory size, wherein the log rate limit value is the maximum number of logs allowed to be processed from the logs generated by the target program.

[0014] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, and the computer program is configured to execute the above-described method for determining the log flow limit value when running.

[0015] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the method for determining the log flow limit value through the computer program.

[0016] This application addresses the issue of log redundancy in a target program during a first time period and the memory size used by the target program to process logs during that first time period, where log redundancy refers to the amount of unprocessed logs. Furthermore, it determines a log rate limit for the target program during a second time period based on the log redundancy and memory size. This rate limit is the maximum number of logs allowed to be processed within the second time period, where the first time period precedes the second time period, and the first and second time periods have the same duration. In other words, by determining the log redundancy and memory size used by the target program during the first time period, the number of logs allowed to be processed within the second time period following the first time period can be determined. This technical solution solves the problem in related technologies where the number of logs input into a program for processing cannot be dynamically limited; it achieves the effect of dynamically limiting the number of logs input into the target program for processing, thus improving the stability of the target program. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the hardware environment for an optional method for determining the log rate limit value according to an embodiment of this application;

[0020] Figure 2 This is a flowchart of an optional method for determining a log rate limit value according to an embodiment of this application;

[0021] Figure 3 This is a schematic diagram of an optional process for determining log redundancy according to an embodiment of this application;

[0022] Figure 4 This is a schematic diagram of system threshold adjustment for an optional method for determining log rate limiting values ​​according to an embodiment of this application;

[0023] Figure 5 This is a structural block diagram of an optional log rate limiting value determination device according to an embodiment of this application. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] According to one aspect of the embodiments of this application, a method for determining a log current limit value is provided. This method for determining the log current limit value is widely applicable to whole-house intelligent digital control application scenarios such as smart homes, smart home ecosystems, and intelligencehouse ecosystems. Optionally, in this embodiment, the above-mentioned method for determining the log current limit value can be applied to, for example... Figure 1 The hardware environment shown consists of multiple terminal devices 102 and a server 104. For example... Figure 1 As shown, server 104 is connected to multiple terminal devices 102 via a network and can be used to provide services (such as application services) to terminals or clients installed on terminals. A database can be set up on the server or independently of the server to provide data storage services for server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data processing services for server 104.

[0027] The aforementioned network may include, but is not limited to, at least one of the following: wired network, wireless network. The aforementioned wired network may include, but is not limited to, at least one of the following: wide area network, metropolitan area network, local area network. The aforementioned wireless network may include, but is not limited to, at least one of the following: Wi-Fi (Wireless Fidelity), Bluetooth. The terminal device 102 may not be limited to PC, mobile phone, tablet computer, smart air conditioner, smart range hood, smart refrigerator, smart oven, smart stove, smart washing machine, smart water heater, smart washing equipment, smart dishwasher, smart projector, smart TV, smart clothes rack, smart curtains, smart audio-visual equipment, smart socket, smart speaker, smart speaker box, smart fresh air equipment, smart kitchen and bathroom equipment, smart bathroom equipment, smart robot vacuum cleaner, smart window cleaning robot, smart mopping robot, smart air purifier, smart steam oven, smart microwave oven, smart water heater, smart air purifier, smart water dispenser, smart door lock, etc.

[0028] This embodiment provides a method for determining log rate limiting values, including but not limited to those applied to computer terminals. Figure 2 This is a flowchart of an optional method for determining a log rate limit value according to an embodiment of this application. The process includes the following steps:

[0029] Step S202: Determine the log redundancy of the target program in the first time period, and the memory size used by the target program to process logs in the first time period, wherein the log redundancy is the amount of logs that the target program has not processed.

[0030] It should be noted that the processing logs mentioned above can be logs generated by the target program.

[0031] Step S204: Determine the log rate limit value of the target program in the second time period based on the log redundancy amount and the memory size, wherein the log rate limit value is the maximum number of logs allowed to be processed for the logs generated by the target program, wherein the first time period is before the second time period, and the first time period and the second time period have the same duration.

[0032] Through the above steps, by determining the log redundancy of the target program in the first time period and the memory size used by the target program to process logs in the first time period, where the log redundancy is the amount of unprocessed logs by the target program, a log rate limit for the target program in the second time period can be determined based on the log redundancy and the memory size. The log rate limit is the maximum number of logs allowed to be processed by the target program, where the first time period precedes the second time period, and the first and second time periods have the same duration. In other words, given the log redundancy and memory size used by the target program in the first time period, the number of logs allowed to be processed by the target program in the second time period following the first time period can be determined based on the log redundancy and the memory size. This technical solution solves the problem in related technologies where it is impossible to dynamically limit the number of logs input into the program for processing; it achieves the effect of dynamically limiting the number of logs input into the target program for processing, thus improving the stability of the target program.

[0033] In one exemplary embodiment, such as Figure 3 As shown, the log redundancy of the target program during the first time period is determined, including:

[0034] Step S302: Determine the log input volume and log processing volume of the target program during the first time period, wherein the log input volume is the number of logs generated by the target program, and the log processing volume is the number of logs actually processed by the target program;

[0035] Step S304: Determine the difference between the log input amount and the log processing amount as the log redundancy amount.

[0036] Understandably, the difference between the number of logs generated and the number of logs processed by the target program within the first time period can be defined as the log redundancy of the target program. For example, if the target program generates 1000 logs and processes 700 logs within the first time period, the log redundancy is 1000 - 700 = 300 logs. The log input and log processing volumes are determined using an accumulation algorithm: for each log generated by the target program, the log input volume increases by 1; for each log processed by the target program, the log processing volume increases by 1. It should be noted that the log processing volume is less than or equal to the log rate limit.

[0037] Regarding the determination of the duration of the first and second time periods: Continuous and uninterrupted reading of the CPU and / or memory would waste the target program's memory. Therefore, a fixed interval can be selected for reading the program's CPU and / or memory. Furthermore, generally, when the amount of logs to be processed in the program is too large, it may cause the program to crash after 5 seconds. Therefore, half of 5 seconds can be selected and rounded down. Thus, the duration of the first and second time periods can be 3 seconds. That is: determine the average crash duration of the target program, and round down half of the average duration to obtain the duration corresponding to the first and second time periods.

[0038] In an exemplary embodiment, determining the memory size used by the target program to process logs during the first time period includes: obtaining the available memory size of the target program during the first time period, and a first weight corresponding to the available memory size; determining the product of the available memory size and the first weight; determining the estimated memory size for the target program to process the log redundancy; and determining the ratio of the available memory size to the product as the memory size used by the target program to process logs during the first time period.

[0039] Optionally, if the processor of the target program cannot be read, the memory size can be determined by the following formula: Memory size = (log input volume - log processing volume) * 0.3M / (current available memory * first weight); where 0.3M is the program memory size occupied by processing a single log, and preferably, the first weight can be 80%.

[0040] In an exemplary embodiment, determining the memory size used by the target program to process logs during the first time period includes: obtaining the processor utilization rate of the target program during the first time period; determining the available memory ratio of the target program during the first time period, wherein the available memory ratio is the ratio of the available memory size of the target program to the total available memory; and determining the memory size used by the target program to process logs during the first time period based on the utilization rate and the available memory ratio.

[0041] In an exemplary embodiment, determining the memory size used by the target program to process logs during the first time period based on the utilization rate and the available memory ratio includes: determining a second weight corresponding to the utilization rate; determining the used memory ratio based on the available memory ratio, and determining a third weight corresponding to the used memory ratio; and determining the sum of the product of the utilization rate and the second weight and the product of the used memory ratio and the third weight as the memory size used by the target program to process logs during the first time period.

[0042] When the processor can be accessed, the memory size is determined by the processor utilization rate and the ratio of used memory, as well as their respective weights. The specific formula is: Memory size = Utilization rate * Second weight + Used memory ratio * Third weight; preferably, the second weight can be 0.6 and the third weight can be 0.8.

[0043] It should be noted that in the above method for determining memory size, total memory is represented by 100% or 1, and the final value of the memory used in the first time period is a percentage or a decimal, such as 70% or 0.7. It should also be noted that the first, second, and third weights mentioned above can be adjusted based on the actual log processing behavior of the target program.

[0044] In an exemplary embodiment, determining the log rate limiting value for the target program in a second time period based on the log redundancy and the memory size includes: determining a first threshold corresponding to the memory size, wherein the first threshold is a memory backlog threshold for the target program; determining the log rate limiting value for the first time period; if the memory size is greater than or equal to the first threshold, determining a first adjustment ratio corresponding to the log rate limiting value for the first time period based on the memory size; and determining the log rate limiting value for the second time period based on the log rate limiting value for the first time period and the first adjustment ratio.

[0045] It should be noted that if the memory usage exceeds the memory backlog threshold during the first time period, a backlog of pending logs may occur, potentially leading to decreased stability and increased susceptibility to crashes in the target program. It should also be noted that, generally, the log rate limit is greater than the log redundancy limit.

[0046] Therefore, when the memory used for log processing exceeds the first threshold within the first time period, the log rate limit for the first time period can be adjusted by a certain ratio, and the adjusted log rate limit for the first time period can be used as the log rate limit for the second time period. Preferably, the first threshold can be 0.9, and the first adjustment ratio can be 0.7. For example, if the log rate limit for the first time period is 1000, and it is determined that the memory used by the target program within the first time period is greater than or equal to 0.9, then the log rate limit for the second time period is: 1000 * 0.7 = 700.

[0047] Through the above embodiments, when the log flow limit is greater than the first threshold, directly adjusting the log flow limit value for the first time period can quickly reduce memory pressure and improve the stability of the target program.

[0048] In an exemplary embodiment, after determining the log rate limit value for the first time period, the method further includes: if the memory size is less than the first threshold, determining a second adjustment ratio corresponding to the log redundancy based on the memory size; and determining the log rate limit value for the second time period based on the log rate limit value for the first time period, the log redundancy, and the second adjustment ratio.

[0049] Optionally, a second threshold and a third threshold can be set, where the second threshold is a memory stability threshold and the third threshold is a memory idle threshold. If the memory used in the first time period is between the second threshold and the first threshold (inclusive), the memory state of the current target program is relatively stable and no adjustment is needed; that is, the log rate limit value for the first time period is determined as the log rate limit value for the second time period. If the memory size is less than the second threshold, it is necessary to further determine whether to adjust the log rate limit value for the second time period based on the log redundancy: if the log redundancy is less than the log redundancy threshold, no adjustment is needed; if the log redundancy is greater than or equal to the log redundancy threshold, the log rate limit value can be appropriately increased. However, to ensure the stability of the target program, the following method can be used: log rate limit value for the first time period + log redundancy * second adjustment ratio. Preferably, the log redundancy threshold can be 20.

[0050] If the memory size is less than the second threshold and the log redundancy is greater than or equal to the log redundancy threshold, and the memory size is between the second threshold and the third threshold, a smaller second adjustment ratio, such as 0.5, can be selected; if the memory size is less than or equal to the third threshold, a larger second adjustment ratio, such as 0.7, can be selected.

[0051] Optionally, after determining the log rate limit value for the target program in the second time period based on the log redundancy and the memory size, the logs generated by the target program can be filtered using a repetition rate algorithm to obtain the log rate limit value; the target program then processes the log rate limit value.

[0052] Obviously, the embodiments described above are only some embodiments of this application, and not all embodiments. To better understand the method for determining the log rate limit value, the following description, in conjunction with embodiments, illustrates the process, but is not intended to limit the technical solutions of the embodiments of this application. Specifically:

[0053] With the development of information technology, more and more business processes require programs to record and process data, resulting in increasingly larger and more numerous log outputs from most programs. An optional embodiment of this application dynamically limits the rate of log output (i.e., controls the amount of logs processed by the target program) while ensuring normal program operation. It should be noted that dynamic rate limiting may result in log loss. Log loss refers to the process of simplifying the logs generated by the target program using algorithms such as repetition rate reduction after determining the log rate limit value.

[0054] It should be noted that the following thresholds and system thresholds are equivalent to the memory size used in the first time period in the above embodiments.

[0055] Furthermore, in optional embodiments of this application, a reasonable output threshold (i.e., the amount of memory used to ensure stable system output) can be calculated based on the CPU and memory usage of the target program, so as to maximize the output of logs while ensuring smooth system operation.

[0056] Furthermore, optional embodiments of this application provide a dynamic current limiting adjustment scheme, including:

[0057] A separate thread is allocated to record the threshold and log redundancy every 3 seconds, where log redundancy = log input volume - log processing volume, and the log redundancy is equivalent to the log redundancy amount in the above embodiment. Based on the threshold and log redundancy every three seconds, the log rate limit can be determined, including:

[0058] 1) When log redundancy > 20 and has reached the user-set maxPerLimit, no adjustment is made.

[0059] It should be noted that users can set maxPerLimit, which is the maximum log rate limit. maxPerLimit represents the maximum amount of logs that the target program can process. With maxPerLimit in place, even if there is log redundancy, there is no need to further increase the log rate limit.

[0060] 2) When log redundancy > 20 and threshold < 0.7, increase log rate limit = log rate limit + log redundancy * 0.7;

[0061] 3) When log redundancy > 20 and 0.7 < threshold < 0.8, increase the log rate limit = log rate limit + log redundancy * 0.5;

[0062] 4) When log redundancy > 20 and 0.8 < threshold < 0.9, increase the log rate limit = log rate limit;

[0063] 5) When log redundancy > 20 and threshold > 0.9, increase log rate limit = log rate limit * 0.7;

[0064] 6) When log redundancy is less than 20 and the threshold is less than 0.9, increase the log rate limit to equal the log rate limit.

[0065] 7) When log redundancy is less than 20 and threshold is greater than 0.9, increase log rate limit by 0.7.

[0066] Figure 4 This is a schematic diagram of a system threshold adjustment method for determining an optional log rate limiting value according to an embodiment of this application, as shown in the diagram. Figure 4 As shown, in short, while ensuring that the log output remains constant, the log rate limit for the second time period is adjusted based on the system threshold for the first time period, thereby adjusting the system threshold for the second time period. Specific solutions include: Solution 1: Decrease the initial maximum value of the system threshold; Solution 2: Increase the initial lower value of the system threshold.

[0067] It should be noted that the system threshold can be set initially or detected in real time for the current time period.

[0068] Furthermore, the following provides a calculation scheme for the threshold:

[0069] Option 1: Calculate the log input and log processing volume per unit time (which can be every 3 seconds);

[0070] Calculation formula: Threshold = (Log input volume - Log processing volume) * 0.3M / (Current available memory * 80%);

[0071] It should be noted that 0.3M is an estimated memory size required by the target program to process a single log entry. If the CPU usage of the target program cannot be read, the threshold can be determined based on the formula in Scheme 1.

[0072] Option 2: Calculate by using the ratio of CPU utilization to available memory (the ratio of current available memory to the total available memory of the program);

[0073] Calculation formula: Threshold = (CPU utilization * 0.6 + Memory utilization * 0.8) * 100%. It should be noted that memory utilization is equivalent to the percentage of memory used in the above embodiment.

[0074] Based on the above solution, the following are specific embodiments of the optional embodiments of this application:

[0075] Assuming a 3-second log redundancy of 100 entries and a current log rate limit of 1000 / s, then based on the above scheme:

[0076] First adjustment: Threshold > 90%: Rate limit = 1000 * 0.7 = 700 / s;

[0077] It should be noted that for the first adjustment: the target program calculates that the threshold within 3 seconds is greater than 90%, and the redundancy within 3 seconds is 100 > 20. Therefore, according to the above: "5) When the log redundancy is > 20 and the threshold is > 0.9, increase the log rate limit value = log rate limit value * 0.7", it can be determined that the log rate limit value for the next 3 seconds corresponding to the first adjustment is the current rate limit value 1000 * 0.7, which is 700 / s. Subsequent adjustments will follow the same principle.

[0078] Second adjustment: Threshold > 90%: Current limit = 700 * 0.7 = 490 / s;

[0079] Third adjustment: 80% < threshold < 90%: current limit = 490 / s;

[0080] Fourth adjustment: 80% < threshold < 90%: current limit = 490 / s;

[0081] Fifth adjustment: Threshold < 70%: Rate limit = 490 + 100 * 0.7 = 560 / s;

[0082] Sixth adjustment: Threshold < 70%: Rate limit = 560 + 100 * 0.7 = 610 / s; (Redundancy remains at 100).

[0083] Seventh adjustment: 70% < threshold < 80%: current limit value = 610 + 60 * 0.5 = 660 / s; (the redundancy remains at 60);

[0084] Eighth adjustment: 70% < threshold < 80%: current limit value = 660 + 40 * 0.5 = 680 / s; (redundancy remains at 40);

[0085] Ninth time: 80% < threshold < 90%: current limit = 680 / s;

[0086] It should be noted that the redundancy values ​​of 100, 60, and 40 mentioned above are all determined according to the formula "Log Redundancy = Log Input Amount - Log Processing Amount". These three values ​​of 100, 60, and 40 are only used as examples.

[0087] In summary, by calculating the system's processing capacity and dynamically adjusting the number of logs processed, the program can maintain smooth operation and protect the optimal data processing under high load.

[0088] Logs are the most important analytical resource after a program exception, and programs are the most important tools for processing work. Maintaining program security and log balance is the most important task to protect. The optional embodiments of this application adopt a dynamic algorithm between log output and program execution to ensure optimal operation of logs and programs.

[0089] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0090] This embodiment also provides a device for determining log rate limiting values. This device is used to implement the above embodiments and preferred embodiments, and details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, hardware implementations, or a combination of software and hardware, are also possible and contemplated.

[0091] Figure 5 This is a structural block diagram of an optional log rate limiting value determination device according to an embodiment of this application. The device includes:

[0092] The first determining module 52 is used to determine the log redundancy of the target program in a first time period and the memory size used by the target program to process logs in the first time period, wherein the log redundancy is the amount of logs that the target program has not processed.

[0093] The second determining module 54 is used to determine the log rate limit value of the target program in the second time period based on the log redundancy amount and the memory size, wherein the log rate limit value is the maximum number of logs that can be processed for the logs generated by the target program, wherein the first time period is before the second time period, and the first time period and the second time period have the same duration.

[0094] By using the aforementioned device, the log redundancy of the target program in a first time period and the memory size used by the target program to process logs in the first time period are determined, where the log redundancy is the amount of unprocessed logs by the target program. Then, a log rate limit for the target program in a second time period can be determined based on the log redundancy and the memory size, where the log rate limit is the maximum number of logs allowed to be processed by the target program. The first time period precedes the second time period, and the first and second time periods have the same duration. In other words, given the log redundancy and memory size used by the target program in the first time period, the number of logs allowed to be processed by the target program in the second time period following the first time period can be determined based on the log redundancy and the memory size. This technical solution solves the problem in related technologies where the number of logs input into a program for processing cannot be dynamically limited; it achieves the effect of dynamically limiting the number of logs input into the target program for processing, thus improving the stability of the target program.

[0095] In an exemplary embodiment, the first determining module 52 is further configured to determine the log input volume and log processing volume of the target program during the first time period, wherein the log input volume is the number of logs generated by the target program, and the log processing volume is the number of logs actually processed by the target program; and the difference between the log input volume and the log processing volume is determined as the log redundancy.

[0096] Understandably, the difference between the number of logs generated and the number of logs processed by the target program within the first time period can be defined as the log redundancy of the target program. For example, if the target program generates 1000 logs and processes 700 logs within the first time period, the log redundancy is 1000 - 700 = 300 logs. The log input and log processing volumes are determined using an accumulation algorithm: for each log generated by the target program, the log input volume increases by 1; for each log processed by the target program, the log processing volume increases by 1. It should be noted that the log processing volume is less than or equal to the log rate limit.

[0097] Regarding the determination of the duration of the first and second time periods: Continuous and uninterrupted reading of the CPU and / or memory would waste the target program's memory. Therefore, a fixed interval can be selected for reading the program's CPU and / or memory. Furthermore, generally, when the amount of logs to be processed in the program is too large, it may cause the program to crash after 5 seconds. Therefore, half of 5 seconds can be selected and rounded down. Thus, the duration of the first and second time periods can be 3 seconds. That is: determine the average crash duration of the target program, and round down half of the average duration to obtain the duration corresponding to the first and second time periods.

[0098] In an exemplary embodiment, the first determining module 52 is further configured to obtain the available memory size of the target program during the first time period, and a first weight corresponding to the available memory size; determine the product of the available memory size and the first weight; determine the estimated memory size for the target program to process the log redundancy; and determine the ratio of the available memory size to the product as the memory size used by the target program to process logs during the first time period.

[0099] Optionally, if the processor of the target program cannot be read, the memory size can be determined by the following formula: Memory size = (log input volume - log processing volume) * 0.3M / (current available memory * first weight); where 0.3M is the program memory size occupied by processing a single log, and preferably, the first weight can be 80%.

[0100] In an exemplary embodiment, the first determining module 52 is further configured to obtain the processor utilization rate of the target program during the first time period; determine the available memory ratio of the target program during the first time period, wherein the available memory ratio is the ratio of the available memory size of the target program to the total available memory; and determine the memory size used by the target program to process logs during the first time period based on the utilization rate and the available memory ratio.

[0101] In an exemplary embodiment, the first determining module 52 is further configured to determine a second weight corresponding to the utilization rate; determine the used memory ratio based on the available memory ratio, and determine a third weight corresponding to the used memory ratio; and determine the sum of the product of the utilization rate and the second weight and the product of the used memory ratio and the third weight as the memory size used by the target program to process logs during the first time period.

[0102] When the processor can be accessed, the memory size is determined by the processor utilization rate and the ratio of used memory, as well as their respective weights. The specific formula is: Memory size = Utilization rate * Second weight + Used memory ratio * Third weight; preferably, the second weight can be 0.6 and the third weight can be 0.8.

[0103] It should be noted that in the above method for determining memory size, total memory is represented by 100% or 1, and the final value of the memory used in the first time period is a percentage or a decimal, such as 70% or 0.7. It should also be noted that the first, second, and third weights mentioned above can be adjusted based on the actual log processing behavior of the target program.

[0104] In an exemplary embodiment, the second determining module 54 is further configured to determine a first threshold corresponding to the memory size, wherein the first threshold is a memory backlog threshold of the target program; determine a log rate limiting value for the first time period; if the memory size is greater than or equal to the first threshold, determine a first adjustment ratio corresponding to the log rate limiting value for the first time period based on the memory size; and determine a log rate limiting value for the second time period based on the log rate limiting value for the first time period and the first adjustment ratio.

[0105] It should be noted that if the memory usage exceeds the memory backlog threshold during the first time period, a backlog of pending logs may occur, potentially leading to decreased stability and increased susceptibility to crashes in the target program. It should also be noted that, generally, the log rate limit is greater than the log redundancy limit.

[0106] Therefore, when the memory used for log processing exceeds the first threshold within the first time period, the log rate limit for the first time period can be adjusted by a certain ratio, and the adjusted log rate limit for the first time period can be used as the log rate limit for the second time period. Preferably, the first threshold can be 0.9, and the first adjustment ratio can be 0.7. For example, if the log rate limit for the first time period is 1000, and it is determined that the memory used by the target program within the first time period is greater than or equal to 0.9, then the log rate limit for the second time period is: 1000 * 0.7 = 700.

[0107] Through the above embodiments, when the log flow limit is greater than the first threshold, directly adjusting the log flow limit value for the first time period can quickly reduce memory pressure and improve the stability of the target program.

[0108] In an exemplary embodiment, the second determining module 54 is further configured to, when the memory size is less than the first threshold, determine a second adjustment ratio corresponding to the log redundancy based on the memory size; and determine a log rate limit value for the second time period based on the log rate limit value for the first time period, the log redundancy, and the second adjustment ratio.

[0109] Optionally, a second threshold and a third threshold can be set, where the second threshold is a memory stability threshold and the third threshold is a memory idle threshold. If the memory used in the first time period is between the second threshold and the first threshold (inclusive), the memory state of the current target program is relatively stable and no adjustment is needed; that is, the log rate limit value for the first time period is determined as the log rate limit value for the second time period. If the memory size is less than the second threshold, it is necessary to further determine whether to adjust the log rate limit value for the second time period based on the log redundancy: if the log redundancy is less than the log redundancy threshold, no adjustment is needed; if the log redundancy is greater than or equal to the log redundancy threshold, the log rate limit value can be appropriately increased. However, to ensure the stability of the target program, the following method can be used: log rate limit value for the first time period + log redundancy * second adjustment ratio. Preferably, the log redundancy threshold can be 20.

[0110] If the memory size is less than the second threshold and the log redundancy is greater than or equal to the log redundancy threshold, and the memory size is between the second threshold and the third threshold, a smaller second adjustment ratio, such as 0.5, can be selected; if the memory size is less than or equal to the third threshold, a larger second adjustment ratio, such as 0.7, can be selected.

[0111] Optionally, the above device further includes a filtering module, which, after determining the log rate limit value of the target program in the second time period based on the log redundancy and the memory size, can filter the logs generated by the target program using a repetition rate algorithm to obtain the log rate limit value; and then the target program processes the log rate limit value.

[0112] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when run.

[0113] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0114] S1, determine the log redundancy of the target program in the first time period, and the memory size used by the target program to process logs in the first time period, wherein the log redundancy is the amount of logs that the target program has not processed.

[0115] S2, determine the log rate limit value of the target program in the second time period based on the log redundancy amount and the memory size, wherein the log rate limit value is the maximum number of logs that can be processed for the logs generated by the target program, wherein the first time period is before the second time period, and the first time period and the second time period have the same duration.

[0116] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0117] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0118] Embodiments of this application also provide an electronic device including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0119] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0120] S1, determine the log redundancy of the target program in the first time period, and the memory size used by the target program to process logs in the first time period, wherein the log redundancy is the amount of logs that the target program has not processed.

[0121] S2, determine the log rate limit value of the target program in the second time period based on the log redundancy amount and the memory size, wherein the log rate limit value is the maximum number of logs that can be processed for the logs generated by the target program, wherein the first time period is before the second time period, and the first time period and the second time period have the same duration.

[0122] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0123] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0124] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0125] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for determining a log rate limit value, characterized in that, include: Determine the log redundancy of the target program in the first time period, and the memory size used by the target program to process logs in the first time period, wherein the log redundancy is the amount of logs that the target program did not process. The log rate limit value for the target program in the second time period is determined based on the log redundancy and the memory size, including: A first threshold corresponding to the memory size is determined, wherein the first threshold is the memory backlog threshold of the target program; Determine the log rate limit value for the first time period; If the memory size is greater than or equal to the first threshold, the first adjustment ratio corresponding to the log flow limit value for the first time period is determined based on the memory size; The log rate limit for the second time period is determined based on the log rate limit for the first time period and the first adjustment ratio, wherein the log rate limit is the maximum number of logs processed for logs generated by the target program, and after determining the log rate limit for the first time period, the method further includes: If the memory size is less than the first threshold, a second adjustment ratio corresponding to the log redundancy is determined based on the memory size; the log rate limit value for the second time period is determined based on the log rate limit value for the first time period, the log redundancy, and the second adjustment ratio, wherein the logs generated by the target program are filtered using a repetition rate algorithm to obtain a log rate limit value of logs; the log rate limit value of logs is processed by the target program.

2. The method of claim 1, wherein, Determine the log redundancy of the target program during the first time period, including: Determine the log input volume and log processing volume of the target program within the first time period, wherein the log input volume is the number of logs generated by the target program, and the log processing volume is the number of logs actually processed by the target program; The difference between the log input volume and the log processing volume is determined as the log redundancy.

3. The method of claim 1, wherein, Determining the memory size used by the target program to process logs during the first time period includes: Obtain the available memory size of the target program during the first time period, and the first weight corresponding to the available memory size; Determine the product of the available memory size and the first weight; Determine the estimated memory size for the target program to process the log redundancy. The ratio of the expected memory size to the product is determined as the memory size used by the target program to process logs during the first time period.

4. The method of claim 1, wherein, Determining the memory size used by the target program to process logs during the first time period includes: Obtain the processor utilization rate of the target program during the first time period; Determine the available memory ratio of the target program during the first time period, wherein the available memory ratio is the ratio of the available memory size of the target program to the total available memory; The amount of memory used by the target program to process logs during the first time period is determined based on the usage rate and the available memory ratio.

5. The method of determining log flow limiting values of claim 4, wherein, Determining the memory size used by the target program to process logs during the first time period based on the usage rate and the available memory ratio includes: Determine the second weight corresponding to the usage rate; The used memory ratio is determined based on the available memory ratio, and a third weight corresponding to the used memory ratio is determined; the sum of the product of the utilization rate and the second weight and the product of the used memory ratio and the third weight is determined as the memory size used by the target program to process logs during the first time period.

6. A device for determining log rate limiting values, characterized in that, include: The first determining module is used to determine the log redundancy of the target program in a first time period, and the memory size used by the target program to process logs in the first time period, wherein the log redundancy is the amount of logs that the target program has not processed. The second determining module is used to determine the log rate limit value for the target program in the second time period based on the log redundancy and the memory size, wherein the log rate limit value is the maximum number of logs allowed to be processed from the logs generated by the target program. The second determining module is further configured to determine a first threshold corresponding to the memory size, wherein the first threshold is a memory backlog threshold of the target program; determine a log rate limit for the first time period; if the memory size is greater than or equal to the first threshold, determine a first adjustment ratio corresponding to the log rate limit for the first time period based on the memory size; and determine a log rate limit for the second time period based on the log rate limit for the first time period and the first adjustment ratio. The second determining module is further configured to, when the memory size is less than the first threshold, determine a second adjustment ratio corresponding to the log redundancy based on the memory size; and determine a log rate limit value for the second time period based on the log rate limit value for the first time period, the log redundancy, and the second adjustment ratio. The device further includes a filtering module, used to filter the logs generated by the target program using a repetition rate algorithm to obtain a log flow limit value of logs; and to process the log flow limit value of logs by the target program.

7. A computer readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs the method of any one of claims 1 to 5. 8.An electronic device comprising a memory and a processor, the electronic device comprising: The memory stores a computer program, and the processor is configured to execute the method of any one of claims 1 to 5 through the computer program.

Citation Information

Patent Citations

  • Log system adjustment and control method, device, storage medium and electronic equipment

    CN111858508A

  • Log collection method and device

    CN112231191A