Data processing methods, apparatus, equipment, storage media and products
By classifying and sampling memory management events, the problem of high resource consumption in memory detection in existing technologies is solved, and efficient and accurate memory detection data generation is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-26
- Publication Date
- 2026-05-26
AI Technical Summary
Existing memory detection methods record data every time a memory management event is triggered, leading to high-frequency CPU resource consumption and causing problems such as device lag and overheating.
When a memory management event is triggered, it is categorized into a first preset category and a second preset category. Based on a preset sampling ratio, it is determined whether to acquire event data. Sampling is performed on the first preset category, and the acquired data is expanded and finally aggregated to generate detection data.
It reduces the consumption of system resources for event data acquisition, improves data acquisition efficiency, and ensures the accuracy of detection data, making it suitable for electronic devices with limited resources.
Smart Images
Figure CN122086697A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of computer technology, and more particularly to data processing methods, apparatus, devices, storage media, and products. Background Technology
[0002] Memory leaks generally refer to the failure or inability to release dynamically allocated memory in a program for some reason, resulting in a waste of system memory, leading to serious consequences such as slowing down program operation or even system crashes.
[0003] When an application experiences memory leaks or memory overflows, developers need to rely on memory-related detection data to troubleshoot the problem. Currently, the conventional memory detection method involves recording data related to each memory allocation and deallocation event (collectively referred to as memory management events) that is triggered, thus obtaining detection data for the memory management events.
[0004] However, memory management events are triggered very frequently, and the above detection methods consume a lot of system resources, such as central processing unit (CPU) resources, which can easily lead to problems such as device lag and overheating. Summary of the Invention
[0005] This disclosure provides data processing methods, apparatus, devices, storage media, and products that can optimize existing data processing solutions for memory detection.
[0006] In a first aspect, embodiments of this disclosure provide a data processing method, including:
[0007] Within a preset time period, in response to a memory management event being triggered, the event category of the memory management event is determined, wherein the memory management event includes memory allocation events and memory release events, and the event category includes a first preset category and a second preset category, wherein the expected triggering frequency of memory management events of the first preset category is greater than the expected triggering frequency of memory management events of the second preset category;
[0008] If the event category of the memory management event is the first preset category, determine whether the memory management event meets the sampling conditions based on the preset sampling ratio. If it does, obtain the event data of the memory management event to obtain the first event data.
[0009] If the event category of the memory management event is the second preset category, the event data of the memory management event is obtained to obtain the second event data;
[0010] The first event data acquired within a preset time period is subjected to a preset augmentation process to obtain target first event data, wherein the preset augmentation process is related to the preset sampling ratio;
[0011] The target first event data and the second event data acquired within the preset time period are aggregated to obtain detection data for the memory management event.
[0012] Secondly, embodiments of this disclosure also provide a data processing apparatus, including:
[0013] An event category determination module is used to determine the event category of a memory management event within a preset time period in response to the triggering of a memory management event. The memory management event includes memory allocation events and memory release events. The event category includes a first preset category and a second preset category. The expected triggering frequency of memory management events in the first preset category is greater than the expected triggering frequency of memory management events in the second preset category.
[0014] The first data acquisition module is used to determine whether the memory management event meets the sampling conditions based on a preset sampling ratio when the event category of the memory management event is the first preset category. If it meets the conditions, the module acquires the event data of the memory management event to obtain the first event data.
[0015] The second data acquisition module is used to acquire event data of the memory management event when the event category of the memory management event is the second preset category, and obtain second event data;
[0016] A data augmentation module is used to perform preset augmentation processing on the first event data acquired within a preset time period to obtain target first event data, wherein the preset augmentation processing is related to the preset sampling ratio;
[0017] The data aggregation module is used to aggregate the target first event data and the second event data acquired within the preset time period to obtain detection data for the memory management event.
[0018] Thirdly, embodiments of this disclosure also provide an electronic device, the electronic device comprising:
[0019] One or more processors;
[0020] Storage device for storing one or more programs.
[0021] When the one or more programs are executed by the one or more processors, the one or more processors implement the data processing method provided in the embodiments of this disclosure.
[0022] Fourthly, embodiments of this disclosure also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the data processing method provided in embodiments of this disclosure.
[0023] Fifthly, embodiments of this disclosure also provide a computer program product, including a computer program that, when executed by a processor, implements the data processing method provided in embodiments of this disclosure.
[0024] The data processing scheme provided in this embodiment determines the event category of a memory management event within a preset time period in response to the triggering of a memory management event. The memory management event includes memory allocation events and memory release events. The event category includes a first preset category and a second preset category. If it is the first preset category, it is determined whether the memory management event meets the sampling conditions based on a preset sampling ratio. If it does, the event data of the memory management event is acquired to obtain first event data. If it is the second preset category, the event data of the memory management event is acquired to obtain second event data. The first event data acquired within the preset time period undergoes a preset expansion process to obtain target first event data. The preset expansion process is related to the preset sampling ratio. The target first event data and the second event data acquired within the preset time period are aggregated to obtain detection data for the memory management event. By adopting the above technical solution, both data acquisition efficiency and the accuracy of memory detection data can be considered. Attached Figure Description
[0025] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.
[0026] Figure 1 This is a schematic flowchart of a data processing method provided in an embodiment of the present disclosure;
[0027] Figure 2 This is a schematic flowchart of another data processing method provided in an embodiment of the present disclosure;
[0028] Figure 3 This is a schematic diagram of an extended processing procedure provided in an embodiment of the present disclosure;
[0029] Figure 4 This is a schematic diagram of the structure of a data processing apparatus provided in an embodiment of the present disclosure;
[0030] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation
[0031] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.
[0032] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.
[0033] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.
[0034] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.
[0035] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".
[0036] Figure 1 This is a schematic diagram of a data processing flow provided by an embodiment of the present disclosure. The embodiments of the present disclosure are applicable to the data processing of memory detection. The method can be executed by a data processing device, which can be implemented in the form of software and / or hardware. Optionally, it can be implemented by an electronic device, which can be a mobile terminal such as a mobile phone, smartwatch, tablet computer, or personal digital assistant, or a personal computer (PC) or server.
[0037] like Figure 1 As shown, the method includes:
[0038] Step 101: Within a preset time period, in response to the triggering of a memory management event, determine the event category of the memory management event, wherein the memory management event includes memory allocation events and memory release events, and the event category includes a first preset category and a second preset category, wherein the expected triggering frequency of memory management events of the first preset category is greater than the expected triggering frequency of memory management events of the second preset category.
[0039] In this embodiment of the disclosure, memory management events include memory allocation events and memory deallocation events. A memory allocation event can be an event where an application requests the operating system to allocate a certain amount of memory space (or memory block), which can be achieved by calling a memory allocation function, such as the `malloc` function. A memory deallocation event can be an event where an application returns previously allocated memory space to the operating system, which can be achieved by calling a memory deallocation function, such as the `dealloc` function. The triggering of memory management events can be detected by a preset detection function, which can be, for example, a callback function, such as the `malloc_logger` function.
[0040] In related technologies, each time a memory management event is detected and triggered, data related to the currently triggered memory management event is recorded. However, the triggering frequency of memory management events is very high, and the above-mentioned indiscriminate data acquisition method consumes a lot of system resources, such as CPU resources, which can easily lead to problems such as electronic devices lag and overheating.
[0041] In this embodiment, memory management events are categorized into a first preset category and a second preset category. The specific criteria for this categorization are not limited; for example, categorization can be based on the size of the managed memory or whether the associated stack is a preset stack (if it is a preset stack, then it belongs to the second preset category). For example, after categorization, the expected trigger frequency of memory management events in the first preset category is guaranteed to be greater than the expected trigger frequency of memory management events in the second preset category. This expected trigger frequency can be understood as the estimated trigger frequency within a preset time period, which can be determined based on the historical trigger frequency over a historical time period of the memory management events. The length of the historical time period can be the same as or different from the length of the preset time period. The historical trigger frequency can be the trigger frequency within a single historical time period or the average of the trigger frequencies over multiple historical time periods. For example, if the preset time is the current day, for memory management events of the first preset category, the historical expected trigger frequency in the past day is 'a', then the expected trigger frequency can be 'a'; for memory management events of the second preset category, the historical expected trigger frequency in the past day is 'b', then the expected trigger frequency can be 'b', where 'a' is greater than 'b'. Specifically, the order of magnitude of 'a' is higher than that of 'b'. For example, 'a' is in the tens of millions and 'b' is in the tens of thousands.
[0042] For example, the start and end times of the preset time can be set according to actual needs. Within the preset time, after a memory management event is detected to be triggered, the event category of the memory management event is first determined, and then it is determined whether the event data of the memory management event needs to be obtained. For example, the event information of the memory management event can be obtained, and the event category can be determined based on the event information. The event information may include information used for category classification, such as the managed memory size or associated stack, etc.
[0043] Step 102: If the event category of the memory management event is the first preset category, determine whether the memory management event meets the sampling conditions based on the preset sampling ratio. If it does, obtain the event data of the memory management event to obtain the first event data.
[0044] In this embodiment of the disclosure, since the expected triggering frequency of the memory management events of the first preset category is relatively high, the number of triggers within the preset time should be higher than the number of triggers of the memory management events of the first preset category. This provides a basis for sampling acquisition, and the event data of the memory management events of the first preset category can be sampled to reduce the number of times the event data of the memory management events of the first preset category is acquired, thereby reducing the excessive consumption of system resources during event data acquisition and improving data acquisition efficiency.
[0045] In this step, if the currently triggered memory management event belongs to the first preset category, it can be determined whether the memory management event meets the sampling conditions. If the sampling conditions are met, the event data of the memory management event is acquired. Optionally, if the sampling conditions are not met, the event data of the memory management event is not acquired. The goal of sampling can be to sample memory management events of the first preset category triggered within a preset time period at a preset sampling ratio, so as to acquire the event data of the memory management events of the first preset category that are sampled. The preset sampling ratio can be set according to actual needs, such as one percent. Assuming that there are 10 million memory management events of the first preset category triggered within a preset time period, the goal is to acquire the event data of 100,000 memory management events of the first preset category. For the current memory management event of the first preset category, it can be determined whether the sampling conditions are met based on the preset sampling ratio. For example, a sampling interval can be set based on a preset sampling ratio, where the sampling interval is the reciprocal of the preset sampling ratio. For example, if the preset sampling ratio is one percent and the sampling interval is 100, the first preset category of memory management events triggered, such as the 1st, 101st, and 201st events, may meet the sampling conditions. Alternatively, a first preset category of memory management event may be randomly selected within the sampling interval as the memory management event that meets the sampling conditions.
[0046] For example, the event data may include information such as memory address identifiers, memory size, and stack information. Specifically, the memory address identifier may be an identifier indicating the starting location of the memory block managed by the memory management event, such as the starting address of an allocated memory block or the starting address of a freed memory block; the memory size may be an identifier indicating the size of the memory block managed by the memory management event, such as the size of an allocated memory block or the size of a freed memory block; the stack information may be information representing the stack corresponding to the memory management event. For memory allocation events, the stack information may be the identifier of the stack requesting memory allocation (which can be denoted as the stack identifier). For memory release events, the event data may not include stack information. The event data of the first preset category of memory management events is recorded as the first event data.
[0047] Step 103: If the event category of the memory management event is the second preset category, obtain the event data of the memory management event to obtain the second event data.
[0048] For example, for memory management events of the second preset category, since the expected triggering frequency is low, in order to ensure the accuracy of the detection data, it is not necessary to obtain sampled event data. There is no need to determine whether the sampling conditions are met. The event data of the memory management events of the second preset category obtained can be directly obtained and recorded as the second event data.
[0049] Step 104: Perform preset expansion processing on the first event data acquired within a preset time period to obtain target first event data, wherein the preset expansion processing is related to the preset sampling ratio.
[0050] For example, when performing preset expansion processing on the first event data, the first event data can be copied according to a preset sampling ratio. The number of copies is the reciprocal of the preset sampling ratio minus 1, thereby obtaining a target number of first event data. The target number is the same as the number of memory management events of the first preset category that are triggered.
[0051] For example, when performing pre-defined augmentation processing on the first event data, the statistical form of the final required detection data can be referenced, and corresponding processing can be performed on the specified types of data in the first event data. For instance, if the detection data needs to include statistical values of numerical data, for numerical data (such as memory size) in the first event data, the data can be divided by a pre-defined sampling ratio to amplify and restore the data, thereby improving the efficiency of subsequent statistics.
[0052] Step 105: Aggregate the target first event data and the second event data acquired within the preset time period to obtain detection data for the memory management event.
[0053] For example, the target first event data and the second event data obtained within a preset time can be aggregated or further processed (such as clustering or merging) to obtain detection data for memory management events. The detection data can be provided to relevant developers for locating, tracing, or analyzing abnormal problems such as memory leaks or memory overflows.
[0054] The data processing method provided in this embodiment determines the event category of the memory management event in response to the triggering of a memory management event within a preset time. The memory management event includes memory allocation events and memory release events. The event category includes a first preset category and a second preset category. If it is the first preset category, it is determined whether the memory management event meets the sampling conditions based on a preset sampling ratio. If it does, the event data of the memory management event is acquired to obtain first event data. If it is the second preset category, the event data of the memory management event is acquired to obtain second event data. The first event data acquired within the preset time is subjected to preset expansion processing to obtain target first event data. The preset expansion processing is related to the preset sampling ratio. The target first event data and the second event data acquired within the preset time are aggregated to obtain detection data for the memory management event. By adopting the above technical solution, memory management events are categorized, and sampled event data is obtained for categories of memory management events expected to trigger more frequently. This reduces excessive consumption of system resources during event data acquisition, improves data acquisition efficiency, and expands the sampled event data when detection data is needed to ensure the accuracy of the detection data. This approach balances data acquisition efficiency and memory detection data accuracy, making it particularly suitable for electronic devices with lower configurations and fewer system resources. In the process of accurately acquiring memory management event data, it minimizes interference with users' normal use of electronic devices and related applications, ensuring a better user experience.
[0055] In some embodiments, the memory size of memory management events in the first preset category is less than or equal to a preset threshold, while the memory size of memory management events in the second preset category is greater than the preset threshold. The advantage of this configuration is that by classifying memory management events based on memory size, the memory size of memory management events in the first preset category is smaller. Therefore, acquiring event data through sampling has less impact on the accuracy of the final detection data, thereby further ensuring the accuracy of the detection data.
[0056] For example, the preset threshold can be set according to the actual situation, such as by analyzing historical event data. For instance, the memory size in the event data of each memory management event obtained within a historical time period can be sorted, such as in ascending order, and the memory size at the target percentile can be determined as the preset threshold. The target percentile can be a large value, such as x%. Analysis shows that the total number of event data is 'a' ten thousand, and the number of historical event data less than or equal to 'y' KB is 'b' ten thousand, accounting for x%. When determining the final detection data, tens of thousands or more data may be merged into one, resulting in a large amount of information redundancy. Therefore, 'y' KB can be determined as the preset threshold. By adopting the sampling acquisition scheme of this embodiment, the event data of these small memory management events is sampled and then pre-expanded in the later stage, which can effectively reduce the system resources consumed during acquisition without affecting the accuracy of the final detection data.
[0057] Since memory management events include memory allocation events and memory deallocation events, and the detection data is mainly used for the analysis of abnormal situations such as memory leaks or memory overflows, it is necessary to know how many allocated memory blocks have not been successfully reclaimed through the detection data. If random sampling is used for any memory management event of the first preset category, it may result in memory allocation events and memory deallocation events corresponding to the same memory block not being sampled at the same time, thus affecting the accuracy of the detection data.
[0058] In some embodiments, determining whether a memory management event meets sampling conditions based on a preset sampling ratio when the event category of the memory management event is the first preset category includes: when the memory management event is a memory allocation event and the corresponding event category is the first preset category, determining whether the memory management event meets sampling conditions based on a preset sampling ratio; when the memory management event is a memory release event and the corresponding event category is the first preset category, determining whether event data of a memory allocation event with the same memory address identifier as the memory release event has been acquired; if so, determining that the memory release event meets sampling conditions. This ensures that memory allocation events and memory release events corresponding to the same memory block can both be sampled, or neither can be sampled, guaranteeing the accuracy of the detection data.
[0059] For example, after determining whether the event data of a memory allocation event with the same memory address identifier as the memory release event has been obtained, the method further includes: if not, determining that the memory release event does not meet the sampling conditions.
[0060] For example, if the currently triggered memory management event of the first preset category is a memory allocation event, it can be determined whether the memory management event meets the sampling conditions based on a preset sampling ratio. If it does, the event data of the memory allocation event is obtained, the corresponding first event data is obtained, and the memory address identifier of the memory allocation event is recorded in a preset data structure. If the currently triggered memory management event of the first preset category is a memory release event, it can be checked whether the memory address identifier of the memory release event is stored in the preset data structure. If it is, it is determined that the memory release event meets the sampling conditions; if not, it is determined that the memory release event does not meet the sampling conditions.
[0061] In some embodiments, determining whether the memory management event meets the sampling conditions based on a preset sampling ratio when the event category of the memory management event is the first preset category includes: calculating the hash value of the memory address identifier of the memory management event when the event category of the memory management event is the first preset category; performing a modulo operation on the hash value to obtain a remainder value; if the remainder value is a preset value, then determining that the memory management event meets the sampling conditions, wherein the ratio of the number of preset values to the number of values within the remainder range of the modulo operation is the preset sampling ratio. Therefore, it is unnecessary to record the memory address identifier of memory allocation events that have already acquired event data, and the comparison operation between the stored memory address identifier and the memory address identifier of the current memory release event is reduced, improving efficiency and enhancing the overall performance of the solution. A detailed description is provided below with reference to specific embodiments.
[0062] Figure 2 This is a flowchart illustrating another data processing method provided in an embodiment of the present disclosure. The embodiments of the present disclosure are optimized based on the various optional solutions in the above embodiments. Specifically, the method includes the following steps:
[0063] Step 201: Within a preset time, in response to a memory management event being triggered, obtain the memory size of the memory management event.
[0064] Memory management events include memory allocation events and memory deallocation events. For example, for a memory allocation event, the return value of the preset detection function may include memory size information, thus allowing the memory size to be obtained. For a memory deallocation event, the return value of the preset detection function may not include memory size information; this information can be obtained by calling a preset system function, such as the malloc_size function.
[0065] Step 202: Determine whether the memory size of the memory management event is less than or equal to the preset threshold. If yes, proceed to step 203; otherwise, proceed to step 207.
[0066] In this embodiment of the disclosure, obtaining the memory size is for determining the event category. The event categories include a first preset category and a second preset category. The expected triggering frequency of memory management events in the first preset category is greater than the expected triggering frequency of memory management events in the second preset category, and the memory size of memory management events in the first preset category is less than or equal to a preset threshold, while the memory size of memory management events in the second preset category is greater than the preset threshold. In this step, if the memory size is determined to be less than or equal to the preset threshold, it is determined to be in the first preset category; otherwise, it is determined to be in the second preset category.
[0067] Step 203: Determine the event category of the memory management event as the first preset category, and calculate the hash value of the memory address identifier of the memory management event.
[0068] For example, memory address identifiers often have numerical characteristics such as even numbers. By calculating a hash value, the problem of uneven distribution of remainders after modulo can be avoided. The hash function used to calculate the hash value can be set according to actual needs and is not limited in specific terms. After processing by the hash function, the resulting hash value is of fixed length, and for the same input, using the same hash function will yield the same hash value. Therefore, for memory allocation events and memory release events with the same memory address identifier, the same hash value can be obtained, leading to the same determination result of whether the sampling conditions are met.
[0069] Step 204: Perform a modulo operation on the hash value to obtain the remainder value.
[0070] The modulus in the modulo operation can be set according to the preset sampling ratio.
[0071] Step 205: Determine if the remainder value is the preset value. If yes, proceed to step 206; otherwise, proceed to step 208.
[0072] The preset sampling ratio is the ratio of the number of preset values to the number of values within the remainder range of the modulo operation. For example, when the modulus is 100, the number of values within the remainder range is 100. If the preset sampling ratio is one percent, the number of preset values is 1. If the preset sampling ratio is one-fiftieth, the number of preset values is 2.
[0073] For example, if the remainder value is not a preset value, it can be determined that the sampling condition is not met, and then step 208 can be executed to determine whether the preset time has ended.
[0074] Step 206: Determine if the memory management event meets the sampling conditions, obtain the event data of the memory management event, obtain the first event data, and execute step 208.
[0075] For example, if the remainder value is not a preset value, it can be determined that the sampling condition is met, the event data of the memory management event is obtained, the first event data is obtained, and then step 208 can be executed to determine whether the preset time has ended.
[0076] Step 207: Determine the event category of the memory management event as the second preset category, obtain the event data of the memory management event, and obtain the second event data.
[0077] For example, if it is determined that the memory size is greater than a preset threshold, it is determined to be the second preset category, and event data can be directly obtained to obtain the second event data.
[0078] Step 208: Determine if the preset time has ended. If yes, proceed to step 209; otherwise, return to step 201.
[0079] If the preset time has expired, subsequent data expansion processing and reporting operations can be performed. Step 209 can be executed immediately after the preset time expires or delayed, depending on actual needs. If the preset time has not expired, it means that new memory management events may be triggered within the preset time. Therefore, the process can return to step 201 to continue detecting the triggering of memory management events.
[0080] Step 209: Perform preset augmentation processing on the first event data acquired within a preset time period to obtain target first event data, wherein the preset augmentation processing is related to the preset sampling ratio.
[0081] Step 210: Aggregate the target first event data and the second event data acquired within a preset time period to obtain detection data for memory management events.
[0082] The data processing method provided in this disclosure classifies memory management events based on the size of the memory block managed by the memory management event. Since the expected triggering frequency of memory management events with small memory is relatively high, the event data of memory management events with small memory can be sampled to reduce excessive consumption of system resources during event data acquisition and improve data acquisition efficiency. When it is necessary to acquire detection data, the sampled event data is expanded to ensure the accuracy of the detection data, thereby balancing data acquisition efficiency and the accuracy of memory detection data. When determining the sampling conditions, the hash value of the memory address identifier of the memory management event is calculated, and the hash value is moduloed. If the remainder obtained is a preset value, it is determined that the sampling conditions are met. This can efficiently and accurately ensure that memory allocation events and memory release events corresponding to the same memory block can both hit the sampling, or neither can hit the sampling, further ensuring the accuracy of the detection data.
[0083] In some embodiments, the aggregation processing of the target first event data and the second event data acquired within the preset time period to obtain detection data for the memory management event includes: merging the target first event data and the second event data acquired within the preset time period that are associated with the same stack to obtain detection data corresponding to different stacks for the memory management event. Therefore, merging event data using the stack as the merging dimension facilitates efficient and targeted problem localization for different stacks.
[0084] For example, the detection data may include multiple sets of data, each set of data including a stack identifier, memory size information corresponding to the stack identifier (such as the cumulative memory size requested, the cumulative memory size released, the cumulative memory size not released, the cumulative requested size of large and small memory, the cumulative released size of large and small memory, the cumulative not released size of large and small memory, etc.), and the cumulative number of memory requests (which can also be further subdivided into the cumulative number of requests for each size of memory, etc.).
[0085] Figure 3 This is a schematic diagram of an extended processing procedure provided in an embodiment of the present disclosure, as shown below. Figure 3 As shown, the detection data (aggregated data) is aggregated according to the stack dimension, such as allocation stack A, allocation stack B, and other allocation stacks. For each allocation stack, taking allocation stack A as an example, for different memory sizes (taking allocation size as an example), such as yKB, zKB, and other allocation sizes, the number of event data entries is counted. Thus, it can be known how many memory allocations allocation stack A has requested in total, how many memory allocations have been requested in total for different allocation sizes, and the total memory size of the memory blocks requested in total by allocation stack A, as well as the total memory size requested for memory blocks of each allocation size (such as yKB) (not shown in the figure).
[0086] In some embodiments, the step of performing a preset expansion process on the first event data acquired within a preset time period to obtain target first event data involves: responding to a preset data aggregation operation, calculating the quotient of the memory size included in the first event data acquired within the preset time period and the preset sampling ratio to obtain a target memory size; counting the number of first event data entries corresponding to different memory sizes associated with the same stack acquired within the preset time period, calculating the quotient of the number of entries and the preset sampling ratio to obtain a target number of entries; and determining the target first event data based on the target memory size and the target number of entries. This reduces data copying operations, improves expansion processing efficiency, and further improves the processing efficiency of subsequent data aggregation processing.
[0087] For example, for a single piece of first event data, the quotient of its memory size and a preset sampling ratio is calculated. If the memory size is yKB and the preset sampling ratio is 1%, the target memory size is 100yKB. Using the stack as the statistical dimension, the number of first event data entries corresponding to different memory sizes associated with the same stack within a preset time period is counted. For example... Figure 3 As shown, the number of first event data entries with an allocation size of yKB associated with allocation stack A is m ten thousand. Calculating the quotient of m ten thousand entries and one percent yields 1 million entries after magnification and restoration. That is, if no sampling is performed, the original 1 million entries are expected. For events of the second preset category, such as those with an allocation size of zKB (greater than yKB), no sampling is performed. Therefore, the number of event data entries in both the acquired data and the aggregated data is n.
[0088] Figure 4 This is a schematic diagram of the structure of a data processing apparatus provided in an embodiment of the present disclosure, as shown below. Figure 4 As shown, the device includes:
[0089] The event category determination module 401 is used to determine the event category of the memory management event in response to the triggering of the memory management event within a preset time. The memory management event includes memory allocation events and memory release events. The event category includes a first preset category and a second preset category. The expected triggering frequency of the memory management event of the first preset category is greater than the expected triggering frequency of the memory management event of the second preset category.
[0090] The first data acquisition module 402 is used to determine whether the memory management event meets the sampling conditions based on a preset sampling ratio when the event category of the memory management event is the first preset category. If it meets the conditions, the module acquires the event data of the memory management event to obtain the first event data.
[0091] The second data acquisition module 403 is used to acquire event data of the memory management event and obtain second event data when the event category of the memory management event is the second preset category;
[0092] The data augmentation module 404 is used to perform preset augmentation processing on the first event data acquired within a preset time to obtain target first event data, wherein the preset augmentation processing is related to the preset sampling ratio;
[0093] The data aggregation module 405 is used to aggregate the target first event data and the second event data acquired within the preset time period to obtain detection data for the memory management event.
[0094] The data processing apparatus provided in this embodiment categorizes memory management events and performs sampled event data acquisition for memory management events of categories expected to trigger more frequently. This reduces excessive consumption of system resources during event data acquisition, improves data acquisition efficiency, and expands the sampled event data when detection data is needed to ensure the accuracy of the detection data. Thus, it balances data acquisition efficiency and memory detection data accuracy, making it particularly suitable for electronic devices with low configuration and limited system resources. In the process of accurately acquiring event data of memory management events, it reduces interference with the user's normal use of electronic devices and related applications, ensuring a good user experience.
[0095] Optionally, the memory size of the memory management event of the first preset category is less than or equal to a preset threshold, and the memory size of the memory management event of the second preset category is greater than the preset threshold.
[0096] Optionally, when the event category of the memory management event is the first preset category, determining whether the memory management event meets the sampling conditions based on a preset sampling ratio includes:
[0097] If the memory management event is a memory allocation event and the corresponding event category is the first preset category, determine whether the memory management event meets the sampling conditions based on the preset sampling ratio.
[0098] If the memory management event is a memory release event and the corresponding event category is the first preset category, determine whether the event data of the memory allocation event with the same memory address identifier as the memory release event has been obtained. If so, determine that the memory release event meets the sampling conditions.
[0099] Optionally, when the event category of the memory management event is the first preset category, determining whether the memory management event meets the sampling conditions based on a preset sampling ratio includes:
[0100] If the event category of the memory management event is the first preset category, calculate the hash value of the memory address identifier of the memory management event;
[0101] Perform a modulo operation on the hash value to obtain the remainder value;
[0102] If the remainder value is a preset value, then the memory management event is determined to meet the sampling condition, wherein the ratio of the number of preset values to the number of values within the remainder range of the modulo operation is the preset sampling ratio.
[0103] Optionally, the data aggregation module is used to: merge the target first event data associated with the same stack and the second event data acquired within the preset time period to obtain detection data corresponding to different stacks for the memory management event.
[0104] Optionally, the data augmentation module includes:
[0105] The target memory size calculation unit is used to respond to a preset data aggregation operation by calculating the quotient of the memory size included in the first event data acquired within a preset time period and the preset sampling ratio to obtain the target memory size.
[0106] The target count calculation unit is used to count the number of the first event data corresponding to different memory sizes associated with the same stack within the preset time period, calculate the quotient of the count and the preset sampling ratio, and obtain the target count.
[0107] The target event data determination unit is used to determine the first event data of the target based on the target memory size and the number of target records.
[0108] The data processing apparatus provided in this disclosure can execute the data processing method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects for executing the method.
[0109] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of this disclosure.
[0110] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Reference is made below. Figure 5 It illustrates an electronic device suitable for implementing embodiments of the present disclosure (e.g., Figure 5 The diagram below shows the structure of the terminal device or server 500. The terminal device in this embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and vehicle terminals (e.g., vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.
[0111] like Figure 5As shown, electronic device 500 may include a processing unit (e.g., central processing unit, graphics processor, etc.) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from storage device 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of electronic device 500. The processing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. An edit / output (I / O) interface 505 is also connected to bus 504.
[0112] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.
[0113] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, it performs the functions defined in the methods of embodiments of this disclosure.
[0114] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.
[0115] The electronic device provided in this embodiment and the data processing method provided in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.
[0116] This disclosure provides a computer storage medium storing a computer program that, when executed by a processor, implements the data processing method provided in the above embodiments.
[0117] This disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the data processing method provided in the above embodiments.
[0118] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.
[0119] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.
[0120] The aforementioned computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device causes the following to occur within a preset time: in response to a memory management event being triggered, the electronic device determines the event category of the memory management event, wherein the memory management event includes memory allocation events and memory release events, and the event category includes a first preset category and a second preset category, wherein the expected triggering frequency of memory management events of the first preset category is greater than the expected triggering frequency of memory management events of the second preset category; if the event category of the memory management event is the first preset category, the electronic device determines whether the memory management event meets the sampling conditions based on a preset sampling ratio; if it does, the electronic device acquires the event data of the memory management event to obtain first event data; if the event category of the memory management event is the second preset category, the electronic device acquires the event data of the memory management event to obtain second event data; the electronic device performs a preset expansion process on the first event data acquired within the preset time to obtain target first event data, wherein the preset expansion process is related to the preset sampling ratio; and the electronic device performs an aggregation process on the target first event data and the second event data acquired within the preset time to obtain detection data for the memory management event.
[0121] Computer program code for performing the operations of this disclosure can be written in one or more programming languages or a combination thereof, including but not limited to object-oriented programming languages such as Java, Smalltalk, and C++, as well as conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0122] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0123] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The names of modules do not necessarily limit the module itself; for example, a data aggregation module can also be described as "a module that aggregates the target first event data and the second event data acquired within the preset time period to obtain detection data for the memory management event."
[0124] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.
[0125] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0126] According to one or more embodiments of this disclosure, a data processing method is provided, comprising:
[0127] Within a preset time period, in response to a memory management event being triggered, the event category of the memory management event is determined, wherein the memory management event includes memory allocation events and memory release events, and the event category includes a first preset category and a second preset category, wherein the expected triggering frequency of memory management events of the first preset category is greater than the expected triggering frequency of memory management events of the second preset category;
[0128] If the event category of the memory management event is the first preset category, determine whether the memory management event meets the sampling conditions based on the preset sampling ratio. If it does, obtain the event data of the memory management event to obtain the first event data.
[0129] If the event category of the memory management event is the second preset category, the event data of the memory management event is obtained to obtain the second event data;
[0130] The first event data acquired within a preset time period is subjected to a preset augmentation process to obtain target first event data, wherein the preset augmentation process is related to the preset sampling ratio;
[0131] The target first event data and the second event data acquired within the preset time period are aggregated to obtain detection data for the memory management event.
[0132] According to one or more embodiments of this disclosure, the memory size of the first preset category of memory management events is less than or equal to a preset threshold, and the memory size of the second preset category of memory management events is greater than the preset threshold.
[0133] According to one or more embodiments of this disclosure, when the event category of the memory management event is the first preset category, determining whether the memory management event meets the sampling conditions based on a preset sampling ratio includes:
[0134] If the memory management event is a memory allocation event and the corresponding event category is the first preset category, determine whether the memory management event meets the sampling conditions based on the preset sampling ratio.
[0135] If the memory management event is a memory release event and the corresponding event category is the first preset category, determine whether the event data of the memory allocation event with the same memory address identifier as the memory release event has been obtained. If so, determine that the memory release event meets the sampling conditions.
[0136] According to one or more embodiments of this disclosure, when the event category of the memory management event is the first preset category, determining whether the memory management event meets the sampling conditions based on a preset sampling ratio includes:
[0137] If the event category of the memory management event is the first preset category, calculate the hash value of the memory address identifier of the memory management event;
[0138] Perform a modulo operation on the hash value to obtain the remainder value;
[0139] If the remainder value is a preset value, then the memory management event is determined to meet the sampling condition, wherein the ratio of the number of preset values to the number of values within the remainder range of the modulo operation is the preset sampling ratio.
[0140] According to one or more embodiments of this disclosure, the aggregation processing of the target first event data and the second event data acquired within the preset time period to obtain detection data for the memory management event includes:
[0141] The target first event data associated with the same stack and the second event data acquired within the preset time period are merged to obtain detection data corresponding to different stacks for the memory management event.
[0142] According to one or more embodiments of this disclosure, the step of performing preset augmentation processing on the first event data acquired within a preset time to obtain target first event data includes:
[0143] In response to a preset data aggregation operation, the target memory size is obtained by calculating the quotient of the memory size included in the first event data acquired within a preset time period and the preset sampling ratio.
[0144] The number of first event data entries corresponding to different memory sizes associated with the same stack within the preset time period is counted, and the quotient of the number of entries and the preset sampling ratio is calculated to obtain the target number of entries.
[0145] The target first event data is determined based on the target memory size and the target number of records.
[0146] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.
[0147] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.
[0148] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.
Claims
1. A data processing method, characterized in that, include: Within a preset time period, in response to a memory management event being triggered, the event category of the memory management event is determined, wherein the memory management event includes memory allocation events and memory release events, and the event category includes a first preset category and a second preset category, wherein the expected triggering frequency of memory management events of the first preset category is greater than the expected triggering frequency of memory management events of the second preset category; If the event category of the memory management event is the first preset category, determine whether the memory management event meets the sampling conditions based on the preset sampling ratio. If it does, obtain the event data of the memory management event to obtain the first event data. If the event category of the memory management event is the second preset category, the event data of the memory management event is obtained to obtain the second event data; The first event data acquired within a preset time period is subjected to a preset augmentation process to obtain target first event data, wherein the preset augmentation process is related to the preset sampling ratio; The target first event data and the second event data acquired within the preset time period are aggregated to obtain detection data for the memory management event.
2. The method according to claim 1, characterized in that, The memory size of the memory management event of the first preset category is less than or equal to a preset threshold, and the memory size of the memory management event of the second preset category is greater than the preset threshold.
3. The method according to claim 1, characterized in that, When the event category of the memory management event is the first preset category, determining whether the memory management event meets the sampling conditions based on a preset sampling ratio includes: If the memory management event is a memory allocation event and the corresponding event category is the first preset category, determine whether the memory management event meets the sampling conditions based on the preset sampling ratio. If the memory management event is a memory release event and the corresponding event category is the first preset category, determine whether the event data of the memory allocation event with the same memory address identifier as the memory release event has been obtained. If so, determine that the memory release event meets the sampling conditions.
4. The method according to claim 1, characterized in that, When the event category of the memory management event is the first preset category, determining whether the memory management event meets the sampling conditions based on a preset sampling ratio includes: If the event category of the memory management event is the first preset category, calculate the hash value of the memory address identifier of the memory management event; Perform a modulo operation on the hash value to obtain the remainder value; If the remainder value is a preset value, then the memory management event is determined to meet the sampling condition, wherein the ratio of the number of preset values to the number of values within the remainder range of the modulo operation is the preset sampling ratio.
5. The method according to claim 1, characterized in that, The aggregation process of the target first event data and the second event data acquired within the preset time period to obtain detection data for the memory management event includes: The target first event data associated with the same stack and the second event data acquired within the preset time period are merged to obtain detection data corresponding to different stacks for the memory management event.
6. The method according to claim 5, characterized in that, The step of performing a preset expansion process on the first event data acquired within a preset time period to obtain target first event data includes: In response to a preset data aggregation operation, the target memory size is obtained by calculating the quotient of the memory size included in the first event data acquired within a preset time period and the preset sampling ratio. The number of first event data entries corresponding to different memory sizes associated with the same stack within the preset time period is counted, and the quotient of the number of entries and the preset sampling ratio is calculated to obtain the target number of entries. The target first event data is determined based on the target memory size and the target number of records.
7. A data processing apparatus, characterized in that, include: An event category determination module is used to determine the event category of a memory management event within a preset time period in response to the triggering of a memory management event. The memory management event includes memory allocation events and memory release events. The event category includes a first preset category and a second preset category. The expected triggering frequency of memory management events in the first preset category is greater than the expected triggering frequency of memory management events in the second preset category. The first data acquisition module is used to determine whether the memory management event meets the sampling conditions based on a preset sampling ratio when the event category of the memory management event is the first preset category. If it meets the conditions, the module acquires the event data of the memory management event to obtain the first event data. The second data acquisition module is used to acquire event data of the memory management event when the event category of the memory management event is the second preset category, and obtain second event data; A data augmentation module is used to perform preset augmentation processing on the first event data acquired within a preset time period to obtain target first event data, wherein the preset augmentation processing is related to the preset sampling ratio; The data aggregation module is used to aggregate the target first event data and the second event data acquired within the preset time period to obtain detection data for the memory management event.
8. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the data processing method as described in any one of claims 1-6.
9. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the data processing method as described in any one of claims 1-6.
10. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the data processing method as described in any one of claims 1-6.