Data processing method and device

Abstract

The invention provides a data processing method and device, and the method comprises the steps: determining at least one business module contained in a target business, determining the resource distribution data corresponding to the at least one business module, and obtaining the resource distribution data according to the business module, and carrying out modular analysis on the resource allocation data subsequently. And generating a data identifier of the resource allocation data, and constructing resource record data based on the resource allocation data and the data identifier. And based on the resource allocation data and the resource record data, generating resource visualization information corresponding to the at least one resource allocation dimension. In the resource visualization information, the current resource allocation and use problems of the target service can be visually determined, and the accuracy and efficiency of data analysis are improved. Modularized analysis is carried out on the resource allocation data and the resource record data, and the data visualization efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and particularly to a data processing method and apparatus. Background Art

[0002] In the field of server development, especially in game server development, memory management is crucial for ensuring system performance and stability. However, the traditional memory allocation and release mechanism has obvious deficiencies in real-time monitoring and diagnosing memory leakage problems. Existing memory monitoring tools usually adopt dynamic library interception technology. Although real-time detection can be achieved, due to its strong generality and lack of optimization for the characteristics of high-frequency memory operations in game servers, the modular analysis ability is limited. The traditional solution is relatively single in the dimension of data analysis and lacks the fine-grained analysis ability classified by functional modules, which greatly increases the difficulty of quick positioning. Therefore, there is an urgent need for a more effective data processing method to solve the above problems. Summary of the Invention

[0003] In view of this, embodiments of this application provide a data processing method to solve the technical defects existing in the prior art. Embodiments of this application also provide a data processing apparatus, a computing device, a computer-readable storage medium, and a computer program product.

[0004] According to the first aspect of the embodiments of this application, a data processing method is provided, including: Determine at least one business module included in a target business, and determine resource allocation data corresponding to the at least one business module; Generate a data identifier of the resource allocation data, and construct resource record data based on the resource allocation data and the data identifier; Generate resource visualization information corresponding to at least one resource allocation dimension based on the resource allocation data and the resource record data.

[0005] Optionally, before determining the resource allocation data corresponding to the at least one business module, it further includes: Assign module tags to the at least one business module, and determine resource request information corresponding to the module tags; The determining the resource allocation data corresponding to the at least one business module includes: Determine the module relationship corresponding to the at least one business module, and determine the resource allocation data corresponding to the at least one business module based on the resource request information and the module relationship.

[0006] Optionally, the assigning module tags to the at least one business module includes: During the compilation phase of the target service, define the module tags for the at least one service module; or, During the running phase of the target service, define the module tags for the at least one service module based on the configuration file of the target service.

[0007] Optionally, the constructing the resource record data based on the resource allocation data and the data identifier includes: Determine the resource allocation metadata of the resource allocation data, and generate resource storage data based on the resource allocation metadata and the data identifier; Determine the mapping relationship between the data identifier and the resource allocation data, and use the mapping relationship and the resource storage data as the resource record data.

[0008] Optionally, the determining the resource allocation data corresponding to the at least one service module includes: Configure the data collection time for the target service; Collect the resource allocation data for the at least one service module according to the data collection time, and asynchronously write the resource allocation data into at least two buffers.

[0009] Optionally, the generating the resource visualization information corresponding to at least one resource allocation dimension based on the resource allocation data and the resource record data includes: Select at least one resource allocation dimension from the service module dimension, time dimension, and resource hierarchy dimension corresponding to the target service; Generate the resource visualization information of the at least one resource allocation dimension based on the resource allocation data and the resource record data.

[0010] Optionally, after generating the resource visualization information corresponding to at least one resource allocation dimension based on the resource allocation data and the resource record data, it further includes: Determine the target area in the resource visualization information; Use the visualization information corresponding to the target area as the resource problem information of the target service.

[0011] Optionally, after generating the resource visualization information corresponding to at least one resource allocation dimension based on the resource allocation data and the resource record data, it further includes: Determine the target resource visualization information of the target service; Compare the resource visualization information with the target resource visualization information, and determine the resource change information of the target service according to the comparison result.

[0012] According to a second aspect of the embodiments of the present application, a data processing device is provided, including: A determination module, configured to determine at least one service module included in a target service, and determine resource allocation data corresponding to the at least one service module; A construction module, configured to generate a data identifier of the resource allocation data, and construct resource record data based on the resource allocation data and the data identifier; A generation module, configured to generate resource visualization information corresponding to at least one resource allocation dimension based on the resource allocation data and the resource record data.

[0013] According to a third aspect of the embodiments of the present application, a computing device is provided, including: A memory and a processor; The memory is used to store computer-executable instructions, and when the processor executes the computer-executable instructions, the steps of the data processing method are implemented.

[0014] According to a fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the data processing method are implemented.

[0015] According to a fifth aspect of the embodiments of the present application, a chip is provided, which stores a computer program, and when the computer program is executed by the chip, the steps of the data processing method are implemented.

[0016] According to a sixth aspect of the embodiments of the present specification, a computer program product is provided, including a computer program or instructions, and when the computer program or instructions are executed by a processor, the steps of the above-mentioned data processing method are implemented.

[0017] The data processing method provided by the present application realizes the acquisition of resource allocation data according to service modules by determining at least one service module included in a target service and determining resource allocation data corresponding to the at least one service module, and then performs modular analysis on the resource allocation data. Generate a data identifier of the resource allocation data, and construct resource record data based on the resource allocation data and the data identifier. Based on the resource allocation data and the resource record data, generate resource visualization information corresponding to at least one resource allocation dimension. From the resource visualization information, the problems existing in the current resource allocation and use of the target service can be intuitively determined, improving the accuracy and efficiency of data analysis. Conduct modular analysis on the resource allocation data and the resource record data to improve the data visualization efficiency. Description of the Drawings

[0018] Figure 1 is a flowchart of a data processing method provided by an embodiment of the present application; Figure 2 It is a resource visualization schematic diagram of a data processing method provided by an embodiment of the present application; Figure 3 It is a processing flow chart of a data processing method applied to game memory monitoring and analysis provided by an embodiment of the present application; Figure 4 It is a structural schematic diagram of a data processing device provided by an embodiment of the present application; Figure 5 It is a structural block diagram of a computing device provided by an embodiment of the present application. Detailed implementation manners

[0019] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the connotation of the present application. Therefore, the present application is not limited by the specific implementations disclosed below.

[0020] The terms used in one or more embodiments of the present application are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of the present application. The singular forms "a", "the", and "said" used in one or more embodiments of the present application and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of the present application refers to and includes any or all possible combinations of one or more of the associated listed items.

[0021] It should be understood that although the terms first, second, etc. may be used in one or more embodiments of the present application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first.

[0022] First, the noun terms related to one or more embodiments of the present invention are explained.

[0023] Macro definition: A term in computer science, mainly used for text replacement or conditional compilation of code in the preprocessing stage. In programming languages such as C and C++, macro definition is implemented through the preprocessing directive #define. It allows programmers to define an alias for constants, code fragments, functions, etc. in the code, thereby simplifying the code, improving readability and maintainability. Flame Graph: A visualization tool used to analyze and display the performance bottlenecks of a program. The flame graph visually shows the call stack information of the program at a certain moment through a graphical method, helping developers quickly locate performance issues. It is generated based on the call stack information of the program, and the call stack records the hierarchical structure and order of function calls. The horizontal axis of the flame graph represents time, and the vertical axis represents the depth of the call stack. The execution time of each function occupies a certain width on the horizontal axis, and the larger the width, the more time the function occupies. Different colors are usually used to distinguish different functions or modules for easy visual identification.

[0024] Double buffering technology: Involves the use of two buffers (or frame buffers). These two buffers are designed to alternately store and access data to achieve smooth data transition and efficient management. The core idea is to separate the data writing (or rendering) operation from the data reading (or display) operation into different buffers, thus avoiding direct conflicts and data competition.

[0025] Hook: In software development, Hook (hook) is a programming mechanism that allows developers to insert custom code or logic when a specific event occurs. Hooks can be used for event interception and processing, that is, Hooks can intercept function calls, message passing, event passing, etc. between software modules. Developers can write custom code in the Hook to modify or extend the behavior of the original program. The Hook mechanism usually does not require modifying the original code to achieve function extension or customization. This makes Hook a low-invasive programming technology, helping to keep the code clean and maintainable.

[0026] In this application, a data processing method is provided. This application also involves a data processing device, a computing device, a computer-readable storage medium, and a computer program product. They will be described in detail one by one in the following embodiments.

[0027] Figure 1 The flowchart of a data processing method provided by an embodiment of this application is shown, which specifically includes the following steps: Step 102: Determine at least one business module included in the target business and determine the resource allocation data corresponding to the at least one business module.

[0028] Specifically, the target service can be a game service corresponding to a game, a database management service, an office service in office software, or an image processing service in image processing software. The operation process of the target service involves the allocation and use of resources such as memory, CPU, and network bandwidth. The service module can be a functional module in the target service. The resource allocation data refers to the resource allocation situation of each service module included in the target service, that is, the allocation situation of resources such as memory, CPU, and network bandwidth. The resource allocation data represents the amount of resources occupied by each service module when the target service is running.

[0029] Based on this, when performing data analysis on the target service, at least one service module included in the target service is determined, and the resource allocation data corresponding to the at least one service module is determined. The resource occupancy situation of each service module in the target service is clarified to facilitate subsequent analysis of the resource allocation and usage situation of the target service in the dimension of the service module.

[0030] Furthermore, considering that at least one service module is included in the target service, in order to facilitate the determination of the resource usage and allocation situation of each service module, a module label can be assigned to each service, and the specific implementation is as follows: Assign module labels to the at least one service module, and determine the resource request information corresponding to the module labels; the determination of the resource allocation data corresponding to the at least one service module includes: determining the module relationship corresponding to the at least one service module, and determining the resource allocation data corresponding to the at least one service module based on the resource request information and the module relationship.

[0031] Specifically, the module label can be a label dynamically bound to the service module. In the case where the target service is a game service, the service modules include, but are not limited to, team formation, database, chat, replay, leaderboard, trading house, player information, etc. Under player information, there can be sub-items: credit score, DIY data, etc. In the case where the target service is an image processing service in image processing software, the service modules include, but are not limited to, basic function modules, special effect processing modules, advanced function modules, professional function modules, and user interface and interaction modules. A label can be assigned to each module. The resource request information corresponding to the module label is the resource request information corresponding to the service module and is used to allocate resources to the service module. The module relationship is the parallel and inclusion relationship between service modules in the target service.

[0032] Based on this, a module label is assigned to each business module in at least one business module to achieve label binding for the business modules. Determine the resource request information of the business module corresponding to the module label. The resource request information represents the size of the resources required for the business module to run and the resource locations that can be allocated to the business module. Determine the module relationships corresponding to at least one business module, and based on the resource request information and the module relationships, determine the resource allocation data corresponding to at least one business module, clarifying the resources allocated to each business module and the locations where the resources are located.

[0033] For example, in the case where the target business is the image processing business in an image processing software, the business modules include, but are not limited to, a basic function module, a special effect processing module, an advanced function module, a professional function module, and a user interface and interaction module. The basic function module can be further divided into image reading and saving, image display, and image editing; the special effect processing module can be further divided into filters and adjustment tools; the advanced function module can be further divided into image analysis, image repair, and image synthesis; the professional function module can be further divided into face recognition, medical image processing, and image processing automation; the user interface and interaction module can be further divided into toolbars and menus, real-time preview, and history and undo. For the basic function module, the special effect processing module, the advanced function module, the professional function module, and the user interface and interaction module, each business module is bound with a label. Record the memory location and memory space size allocated to each business module.

[0034] In summary, determining the module relationships corresponding to at least one business module and determining the resource allocation data corresponding to at least one business module based on the resource request information and the module relationships facilitate subsequent storage and analysis of the resource allocation data according to the business modules.

[0035] Furthermore, when binding the module label to the business module, it can be bound during the compilation stage of the target business or during the running stage of the target business. The specific implementation is as follows: During the compilation stage of the target business, define the module label for the at least one business module; or, during the running stage of the target business, based on the configuration file of the target business, define the module label for the at least one business module.

[0036] Based on this, during the compilation stage of the target business, module tags are defined for at least one business module, and the module tags are bound to the business modules in the form of macro definitions. The corresponding relationship between the module tags and the business modules is determined during the code writing stage, which has high stability and efficiency. Or, during the running stage of the target business, based on the configuration file of the target business, module tags are defined for at least one business module. When the program of the target business runs, the tags are injected into the business modules through the configuration file or dynamic parameters. This method provides higher flexibility and allows changing the corresponding relationship between the module tags and the business modules without modifying the code.

[0037] In summary, during the compilation stage of the target business, or during the running stage of the target business, module tags are defined for at least one business module to achieve diverse definitions of module tags.

[0038] Furthermore, when processing data for the target business, a multi-threaded asynchronous processing mechanism can be adopted to improve data processing efficiency. The specific implementation is as follows: Configure the data collection time for the target business; collect the resource allocation data for the at least one business module according to the data collection time, and the resource allocation data is asynchronously written into at least two buffers.

[0039] Based on this, the data collection time can be the time interval for data collection of the target business. After configuring the data collection time for the target business, the resource allocation data can be collected for at least one business module according to the data collection time. The resource allocation data is asynchronously written into at least two buffers.

[0040] In practical applications, an independent monitoring thread can be set up. The independent monitoring thread copies the memory data related to the target business from the lock-free circular buffer at a configurable time interval and asynchronously writes it into a file to avoid blocking the main thread. The double-buffer technology is adopted to ensure the atomicity and integrity during the data synchronization process.

[0041] In summary, collecting the resource allocation data for at least one business module according to the data collection time, and asynchronously writing the resource allocation data into at least two buffers can improve the performance of the multi-threaded asynchronous processing mechanism, improve data processing efficiency, and ensure the atomicity and integrity of the data during the data synchronization process.

[0042] Step 104: Generate a data identifier for the resource allocation data, and construct resource record data based on the resource allocation data and the data identifier.

[0043] Specifically, after determining at least one business module included in the target business and determining the resource allocation data corresponding to the at least one business module, a data identifier of the resource allocation data can be generated, and resource record data can be constructed based on the resource allocation data and the data identifier. The data identifier of the resource allocation data is used for retrieving and verifying the resource allocation data, and the data identifier can represent the uniqueness of the resource allocation data. The data identifier can be obtained by processing the resource allocation data using data marking algorithms such as hash algorithms, pseudo-random number generation algorithms, unique identifier generation algorithms, and checksum algorithms for data retrieval and verification. The resource record data refers to the data obtained by mapping and compressing the resource allocation data and the data identifier, and the resource record data can be stored in the form of a dictionary file or a hash table.

[0044] Based on this, after determining at least one business module included in the target business and determining the resource allocation data corresponding to the at least one business module, the data marking algorithm can be used for the resource allocation data to generate the data identifier of the resource allocation data. Construct resource record data based on the resource allocation data and the data identifier to achieve the purpose of compressing and storing the resource allocation data and the data identifier, and reduce the occupation of resources such as memory, CPU, and bandwidth during the operation of the target business.

[0045] Furthermore, after obtaining the resource allocation data, the resource allocation data can be compressed and abstracted to facilitate data management and analysis. The specific implementation is as follows: Determine the resource allocation metadata of the resource allocation data, and generate resource storage data based on the resource allocation metadata and the data identifier; determine the mapping relationship between the data identifier and the resource allocation data, and use the mapping relationship and the resource storage data as the resource record data.

[0046] Specifically, the resource allocation metadata is the metadata of the resource allocation data. The resource allocation metadata includes the data identifier corresponding to the resource allocation data and the size of the resource. The resource storage data includes the resource allocation metadata and the identifier corresponding to the resource allocation metadata. The mapping relationship between the data identifier and the resource allocation data represents the corresponding relationship between the data identifier and the resource allocation data. Each business module can correspond to a data identifier and a piece of resource allocation data, so there is a one-to-one correspondence between the resource allocation data and the data identifier under the business module, that is, the mapping relationship. The resource allocation data can be data stored in the form of a stack, that is, the memory allocation stack information, and the mapping relationship is the mapping relationship between the memory allocation stack information and the data identifier.

[0047] Based on this, determine the resource allocation metadata for the resource allocation data. The resource allocation metadata includes the data identifier corresponding to the resource allocation data and the size of the resource. Compose the resource storage data based on the resource allocation metadata and the data identifier. Determine the mapping relationship between the data identifier and the resource allocation data, and use the mapping relationship and the resource storage data as the resource record data. The mapping relationship between the data identifier and the resource allocation data can be stored in the form of a dictionary file.

[0048] Continuing with the above example, the target service is the image processing service in the image processing software. After obtaining the memory resource allocation data corresponding to at least one functional module in the image processing service, determine the metadata of the memory resource allocation data, that is, the size of the memory allocation and the hash value obtained by performing a hash calculation on the memory resource allocation data. Store the mapping relationship between the hash value and the memory resource allocation data as an independent dictionary file. Store the dictionary file, the metadata of the memory resource allocation data, and the hash value of the memory resource allocation data as the resource record data.

[0049] In summary, use the mapping relationship and the resource storage data as the resource record data to compress and abstract the resource allocation data, facilitating data management and analysis and reducing the memory occupancy during the operation of the target service.

[0050] Step 106: Generate resource visualization information corresponding to at least one resource allocation dimension based on the resource allocation data and the resource record data.

[0051] Specifically, after generating the data identifier of the resource allocation data and constructing the resource record data based on the resource allocation data and the data identifier, resource visualization information corresponding to at least one resource allocation dimension can be generated based on the resource allocation data and the resource record data. Among them, the resource visualization information can be the resource allocation data and the resource record data presented in the form of graphics, images, animations, or other visual elements.

[0052] Based on this, after generating the data identifier of the resource allocation data and constructing the resource record data based on the resource allocation data and the data identifier, perform visualization processing on the resource allocation data and the resource record data to generate resource visualization information in the form of graphics, images, animations, or other visual elements corresponding to at least one resource allocation dimension, facilitating the analysis of the resource allocation data and the resource record data.

[0053] Furthermore, considering that when collecting data for the target service, the resource allocation data is collected for at least one service module, when visualizing the resource allocation data and the resource record data later, resource visualization information corresponding to at least one resource allocation dimension can also be generated. The specific implementation is as follows: Select at least one resource allocation dimension from the business module dimension, time dimension, and resource hierarchy dimension corresponding to the target business; generate the resource visualization information for the at least one resource allocation dimension based on the resource allocation data and the resource record data.

[0054] Specifically, the business module dimension refers to the business module of the target business, which is used to label the business module information of the target business in the resource visualization information; the time dimension refers to the time point when data is collected for the target business, which is used to display the data collection time in the resource visualization information; the resource hierarchy dimension refers to the stack storage hierarchy of the collected resource allocation data, which is used to display data in the resource visualization information according to the stack hierarchy division.

[0055] Based on this, select at least one resource allocation dimension from the business module dimension, time dimension, and resource hierarchy dimension corresponding to the target business. Generate the resource visualization information for the at least one resource allocation dimension based on the resource allocation data and the resource record data, and intuitively display the hot areas in the resource allocation data and the resource record data.

[0056] Continuing with the above example, after obtaining the dictionary file, the metadata of the memory resource allocation data, and the hash value of the memory resource allocation data, visualization can be performed on the dictionary file and the resource allocation data. Based on the dictionary file and the resource allocation data, draw a flame graph as shown in Figure 2 The flame graph shows the resource visualization information. By analyzing the resource visualization information, the hot area of memory in the image processing business can be determined.

[0057] In summary, generate the resource visualization information for the at least one resource allocation dimension based on the resource allocation data and the resource record data, improve the information richness of the resource visualization information, and can analyze the resource visualization information from multiple perspectives.

[0058] Furthermore, the resource visualization information can intuitively display the resource usage of the target business, and the resource problems existing in the target business can be determined by analyzing the resource visualization information. The specific implementation is as follows: Determine the target area in the resource visualization information; use the visualization information corresponding to the target area as the resource problem information of the target business.

[0059] Based on this, the target area represents the hotspot area of ​​resource usage in the resource visualization information. Determine the target area with high resource usage in the resource visualization information. Use the visualization information corresponding to the target area as the resource problem information of the target business. The target area represents the performance hotspot area of ​​the target business. During the time period of the target area, the resource usage is high, which is the resource usage hotspot area of ​​the target business. After the program of the target business is finished running, it is found that some of the requested memory and other resources are still not released, which is displayed in the resource visualization information. At this time, you can check whether there is a memory leak problem, which is convenient for managing and optimizing the resource usage of the target business.

[0060] Using the above example, Figure 2 In the flame graph shown, the solid-line box represents the target area, that is, the hotspot area of ​​memory usage in the image processing business. The hotspot area indicates that the image processing business has a resource usage hotspot problem during the time interval. In the target area, the dotted-line box area indicates that after the image processing business ends, some of the applied content has not been released. Therefore, it is determined that the dotted-line box area has a memory leak problem and needs to be processed in time. The hash value can be used to reversely match the stack to locate the content growth point in the image processing business operation cycle or iterative version.

[0061] In summary, the visualization information corresponding to the target area is used as the resource problem information of the target business. By analyzing the resource problem information, the problems existing in the resource use process of the target business can be determined, and then the problems can be solved in a targeted manner.

[0062] Furthermore, considering that data can be collected for the target business at fixed intervals, when visualizing the collected resource allocation data, visualization processing can be performed based on the resource allocation data collected at multiple time points, so that the difference comparison of multiple resource visualization information can be performed, which is specifically implemented as follows: Determine target resource visualization information of the target business; compare the resource visualization information with the target resource visualization information, and determine resource change information of the target business according to the comparison result.

[0063] Based on this, the target resource visualization information of the target business can be the resource allocation data obtained by collecting data for the target business at a time point before or after collecting resource allocation data, as well as the resource visualization information obtained by data visualization after resource recording data. The target resource visualization information and the resource visualization information can be resource visualization information generated at different data collection moments. The resource visualization information is compared with the target resource visualization information, and the resource change information of the target business is determined based on the comparison results to understand the resource usage changes at different moments during the operation of the target business.

[0064] In summary, by comparing the resource visualization information with the target resource visualization information, the differential comparison of at least two pieces of resource visualization information is realized, so as to obtain the changes in resource allocation and resource usage of the target service at different times.

[0065] The data processing method provided by this application determines at least one service module included in the target service, and determines the resource allocation data corresponding to at least one service module, so as to realize the acquisition of resource allocation data according to the service module, and then perform modular analysis on the resource allocation data. Generate a data identifier of the resource allocation data, and construct resource record data based on the resource allocation data and the data identifier. Based on the resource allocation data and the resource record data, generate resource visualization information corresponding to at least one resource allocation dimension. From the resource visualization information, the problems existing in the current resource allocation and usage of the target service can be intuitively determined, improving the accuracy and efficiency of data analysis. Conduct modular analysis on the resource allocation data and the resource record data to improve the data visualization efficiency.

[0066] The following combines the attached Figure 3 Taking the game memory monitoring and analysis application as an example of the data processing method provided by this application, the data processing method will be further described. Among them, Figure 3 FIG. shows a processing flow chart of a data processing method applied to game memory monitoring and analysis provided by an embodiment of this application, which specifically includes the following steps: Step 302: Bind a module label according to the function module of the target game.

[0067] In practical applications, for the target game, in the context of a C++ server, the function modules of the game server include but are not limited to teaming, chatting, replay, leaderboard, trading house, etc. The player information module of the target game also includes sub-items such as credit score and DIY data. For the function module of DIY data, the label DIY_MODULE represents the player's DIY-related data. When a player modifies their DIY data on the client and uploads it to the server, it will trigger a call link across two modules, namely creating new player DIY data - database saving. However, the memory allocation on this chain will all be labeled with DIY_MODULE, indicating that this round of memory allocation is caused by the player's DIY data. When viewing the final data in the flame graph, the allocated memory stack can be viewed separately according to the label.

[0068] In specific implementation, when binding module tags to functional modules, the originating module principle is followed. That is, if the memory allocation is triggered by a request from module A, it is labeled with the tag of module A, even if subsequent calls involve other modules (such as a database). For example, when a player modifies DIY data and triggers database saving, all relevant memory allocations are labeled as DIY_MODULE. The tag binding mechanism supports static binding and dynamic inheritance. Among them, static binding is to directly embed a tag macro in the code, such as new MEM_TAG(DIY_MODULE) PlayerData(); dynamic inheritance is to pass the module context through thread-local storage (TLS). For example, when the DIY module calls the database interface, the memory allocations inside the database operation automatically inherit the DIY_MODULE tag, ensuring that the memory ownership of the cross-module link is traceable.

[0069] Step 304: Record the memory allocation location of the functional module and the attribution relationship of the functional module.

[0070] In practical applications, each time memory is allocated, a dictionary can be used with the handle of the memory allocation as the key to record information such as the tag of the current environment, the stack hash value, and the size of the memory allocation during this application. Among them, the dictionary is a thread-safe lock-free hash table, and the handle is a digest hash compressed from the address of the originating call stack function. When the memory is released, the deletion of the hash table entry is automatically triggered, and the reference counting mechanism is used to ensure that asynchronous persistent threads do not access the released address.

[0071] Step 306: Collect memory allocation stack information according to functional modules for the target game.

[0072] Step 308: Use a hash algorithm to generate the hash value of the memory allocation stack information, and store the hash value and the metadata of the memory allocation stack information while the target game is running.

[0073] Step 310: Generate a dictionary file based on the mapping relationship between the hash value and the memory allocation stack information.

[0074] In specific implementation, for each allocated memory, record the addresses of the first 32 layers of entry functions, compress them into a hash value, and represent it in hexadecimal. At the same time, maintain a dictionary file. When querying the dictionary using the hash value, each different hash value can obtain the corresponding detailed function stack at that time. The dictionary file records the call chain stack information from the top layer to the bottom layer.

[0075] In practical applications, the first 64 layers of the stack during memory allocation can be collected (covering more than 99% of the game logic paths), and the excess part is truncated to reduce the hash calculation overhead, generating a 128-bit digest hash compressed from the stack function addresses. During operation, only the hash value, memory size, and module label are stored. When the server terminates, a mapping dictionary file of the hash value and the original stack is output, including the function address, symbol name, and source code line number of each layer.

[0076] Step 312: Generate a flame graph based on the dictionary file and the memory allocation stack information, and determine the memory hot spot area in the flame graph.

[0077] In specific implementation, by parsing the memory record file corresponding to the dictionary file and the memory allocation stack information, a memory usage flame graph divided by module, time, or stack level is generated to visually display the hot spot area. At the same time, it also supports the differential comparison of multiple record files. By reverse matching the stack through the hash value, the memory growth points in the target game version iteration or operation cycle are located.

[0078] In addition, through a multi-threaded asynchronous processing mechanism, an independent monitoring thread is configured. The independent monitoring thread copies memory data from the lock-free circular buffer at a configurable time interval (such as every 5 seconds) and asynchronously writes it to a file to avoid blocking the main thread. The double-buffer technology is adopted to ensure the atomicity and integrity during the data synchronization process.

[0079] In summary, the data processing method provided by an embodiment of this specification realizes a higher-precision memory attribution analysis through module labels, hash compression, and game scene optimization, ensuring the high real-time performance of the game server, and improving the diagnosis efficiency through module-stack joint analysis. A low-intrusive Hook and a data synchronization mechanism triggered by heartbeat cycles are designed according to the characteristics of high-frequency memory operations in game logic, and the performance loss is less than 2%. Through the association of labels and hash stacks, it supports two-dimensional analysis of "statistical total memory by module" and "trace leakage points within the module by stack". Only lightweight data is recorded during operation, and complete information is restored in combination with the dictionary during the offline stage, balancing performance and diagnostic depth.

[0080] Corresponding to the above method embodiment, this application also provides an embodiment of a data processing device. Figure 4 The structural schematic diagram of a data processing device provided by an embodiment of this application is shown. As Figure 4 shown, the device includes: A determination module 402, configured to determine at least one service module included in the target service, and determine the resource allocation data corresponding to the at least one service module; A construction module 404, configured to generate a data identifier of the resource allocation data, and construct resource record data based on the resource allocation data and the data identifier; A generation module 406, configured to generate resource visualization information corresponding to at least one resource allocation dimension based on the resource allocation data and the resource record data.

[0081] In an optional embodiment, the determination module 402 is further configured to: Assign module tags to the at least one service module and determine the resource request information corresponding to the module tags; The determining the resource allocation data corresponding to the at least one service module includes: Determine the module relationships corresponding to the at least one service module, and determine the resource allocation data corresponding to the at least one service module based on the resource request information and the module relationships.

[0082] In an optional embodiment, the determination module 402 is further configured to: Define the module tags for the at least one service module during the compilation stage of the target service; or, Define the module tags for the at least one service module based on the configuration file of the target service during the running stage of the target service.

[0083] In an optional embodiment, the construction module 404 is further configured to: Determine the resource allocation metadata of the resource allocation data, and generate resource storage data based on the resource allocation metadata and the data identifier; Determine the mapping relationship between the data identifier and the resource allocation data, and use the mapping relationship and the resource storage data as the resource record data.

[0084] In an optional embodiment, the determination module 402 is further configured to: Configure a data collection time for the target service; Collect the resource allocation data for the at least one service module according to the data collection time, and asynchronously write the resource allocation data into at least two buffers.

[0085] In an optional embodiment, the generation module 406 is further configured to: Select at least one resource allocation dimension from the service module dimension, time dimension, and resource hierarchy dimension corresponding to the target service; Generate the resource visualization information of the at least one resource allocation dimension based on the resource allocation data and the resource record data.

[0086] In an optional embodiment, the generation module 406 is further configured to: Determine a target area in the resource visualization information; Use the visualization information corresponding to the target area as the resource problem information of the target service.

[0087] In an optional embodiment, the generation module 406 is further configured to: Determine the target resource visualization information of the target service; Compare the resource visualization information with the target resource visualization information, and determine the resource change information of the target service according to the comparison result.

[0088] The data processing device provided by the present application realizes the acquisition of resource allocation data according to service modules by determining at least one service module included in the target service and determining the resource allocation data corresponding to the at least one service module, and then performs modular analysis on the resource allocation data. Generate a data identifier of the resource allocation data, and construct resource record data based on the resource allocation data and the data identifier. Based on the resource allocation data and the resource record data, generate resource visualization information corresponding to at least one resource allocation dimension. From the resource visualization information, it is possible to intuitively determine the problems existing in the current resource allocation and use of the target service, improving the accuracy and efficiency of data analysis. Performing modular analysis on the resource allocation data and the resource record data improves the data visualization efficiency.

[0089] The above is a schematic solution of a data processing device according to this embodiment. It should be noted that the technical solution of this data processing device and the technical solution of the above data processing method belong to the same concept. For the details not described in detail in the technical solution of the data processing device, reference can be made to the description of the technical solution of the above data processing method. In addition, each component in the device embodiment should be understood as a functional module that must be established to implement each step of the program flow or each step of the method. The device claims defined by such a group of functional modules should be understood as mainly realizing the functional module architecture of the solution through the computer program recorded in the specification, rather than mainly realizing the physical device of the solution through hardware means.

[0090] Figure 5 FIG. shows a structural block diagram of a computing device 500 according to an embodiment of the present application. The components of the computing device 500 include, but are not limited to, a memory 510 and a processor 520. The processor 520 is connected to the memory 510 through a bus 530, and the database 550 is used to store data.

[0091] The computing device 500 also includes an access device 540, which enables the computing device 500 to communicate via one or more networks 560. Examples of such networks include the Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device 540 may include one or more of any type of wired or wireless network interface (e.g., Network Interface Card (NIC)), such as an IEEE802.11 Wireless Local Area Network (WLAN) wireless interface, Worldwide Interoperability for Microwave Access (Wi-MAX) interface, Ethernet interface, Universal Serial Bus (USB) interface, cellular network interface, Bluetooth interface, Near Field Communication (NFC) interface, and so on.

[0092] In one embodiment of the present application, the above components of the computing device 500 and Figure 5 other components not shown therein may also be connected to each other, for example, via a bus. It should be understood that Figure 5 the block diagram of the computing device shown is for illustrative purposes only and is not a limitation on the scope of the present application. Those skilled in the art can add or replace other components as needed.

[0093] The computing device 500 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or PCs. The computing device 500 can also be a mobile or stationary server.

[0094] Wherein, the processor 520 is used to execute computer-executable instructions of the data processing method.

[0095] The above is a schematic solution of a computing device in this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the above data processing method belong to the same concept. For the details not described in the technical solution of the computing device, reference can be made to the description of the technical solution of the above data processing method.

[0096] One embodiment of the present application also provides a computer-readable storage medium, which stores computer instructions that are executed by a processor for a data processing method.

[0097] The above is a schematic solution of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the above data processing method belong to the same concept. For the details not described in detail in the technical solution of the storage medium, reference can be made to the description of the technical solution of the above data processing method.

[0098] An embodiment of the present application further provides a chip, which stores a computer program. When the computer program is executed by the chip, the steps of the data processing method are implemented.

[0099] An embodiment of this specification further provides a computer program product, including a computer program or instruction. When the computer program or instruction is executed by a processor, the steps of the above data processing method are implemented.

[0100] The above is a schematic solution of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the above data processing method belong to the same concept. For the details not described in detail in the technical solution of the computer program product, reference can be made to the description of the technical solution of the above data processing method.

[0101] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the figures do not necessarily require the particular order or sequential order shown to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0102] The computer instructions include computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0103] It should be noted that, for the foregoing method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should know that this application is not limited by the described action sequence, because according to this application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0104] In the above embodiments, the descriptions of the respective embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0105] The preferred embodiments of the present application disclosed above are only used to help explain the present application. The optional embodiments do not describe all the details in detail, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made according to the content of this application. The present application selects and specifically describes these embodiments to better explain the principle and practical application of the present application, so that those skilled in the art can well understand and utilize the present application. The present application is only limited by the claims and their full scope and equivalents.

Claims

1. A data processing method, characterized in that: include: Determine at least one service module included in the target service, and determine resource allocation data corresponding to the at least one service module; generating a data identifier of the resource allocation data, and constructing resource record data based on the resource allocation data and the data identifier; Resource visualization information corresponding to at least one resource allocation dimension is generated based on the resource allocation data and the resource record data.

2. The data processing method according to claim 1, characterized in that: Before determining the resource allocation data corresponding to the at least one service module, the method further includes: Allocating a module tag to the at least one business module, and determining resource request information corresponding to the module tag; The determining resource allocation data corresponding to the at least one service module includes: A module relationship corresponding to the at least one business module is determined, and the resource allocation data corresponding to the at least one business module is determined based on the resource request information and the module relationship.

3. The data processing method according to claim 2, characterized in that: The assigning a module label to the at least one service module includes: In the compilation phase of the target service, the module tag is defined for the at least one service module; or, During the operation phase of the target service, the module tag is defined for the at least one service module based on a configuration file of the target service.

4. The data processing method according to claim 1, characterized in that: The constructing resource record data based on the resource allocation data and the data identifier includes: Determining resource allocation metadata of the resource allocation data, and generating resource storage data based on the resource allocation metadata and the data identifier; A mapping relationship between the data identifier and the resource allocation data is determined, and the mapping relationship and the resource storage data are used as the resource record data.

5. The data processing method according to claim 1, characterized in that: The determining resource allocation data corresponding to the at least one service module includes: Configure data collection time for the target business; The resource allocation data is collected for the at least one service module according to the data collection time, and the resource allocation data is asynchronously written into at least two buffers.

6. The data processing method according to claim 1, characterized in that: The generating, based on the resource allocation data and the resource record data, resource visualization information corresponding to at least one resource allocation dimension comprises: Selecting at least one resource allocation dimension from among a business module dimension, a time dimension, and a resource level dimension corresponding to the target business; The resource visualization information of the at least one resource allocation dimension is generated based on the resource allocation data and the resource record data.

7. The data processing method according to claim 1, characterized in that: After generating resource visualization information corresponding to at least one resource allocation dimension based on the resource allocation data and the resource record data, the method further includes: determining a target area in the resource visualization information; The visualization information corresponding to the target area is used as resource problem information of the target business.

8. The data processing method according to claim 1, characterized in that: After generating resource visualization information corresponding to at least one resource allocation dimension based on the resource allocation data and the resource record data, the method further includes: Determine target resource visualization information of the target business; The resource visualization information is compared with the target resource visualization information, and resource change information of the target business is determined according to the comparison result.

9. A data processing device, characterized in that: include: A determination module, configured to determine at least one service module included in the target service, and determine resource allocation data corresponding to the at least one service module; A construction module, configured to generate a data identifier of the resource allocation data, and construct resource record data based on the resource allocation data and the data identifier; The generating module is configured to generate resource visualization information corresponding to at least one resource allocation dimension based on the resource allocation data and the resource record data.

10. A computing device, characterized in that: include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the steps of the data processing method according to any one of claims 1 to 8.

11. A computer-readable storage medium storing computer instructions, characterized in that: When the instruction is executed by a processor, the steps of the data processing method described in any one of claims 1 to 8 are implemented.

12. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the data processing method according to any one of claims 1 to 8 are implemented.

Images