Data processing method and system

Abstract

The invention provides a data processing method and system, and the method comprises the steps: responding to an access request carrying a target resource identifier, and querying a virtual address mapping relation corresponding to the target resource identifier; under the condition that the virtual address mapping relationship is not queried, determining a target resource type according to the target resource identifier, and determining an allocated target virtual address from a logic memory pool corresponding to the target resource type, the logic memory pool being obtained by performing resource type division in a virtual address space; a target memory allocation interface is called, a target resource corresponding to the target resource identifier is loaded to a target physical memory, a mapping relation between the target physical memory and an allocated target virtual address is established, a virtual address handle corresponding to the target virtual address is generated, and the target memory allocation interface integrates memory allocation interfaces of all host platforms; accessing the target resource according to the virtual address handle; through the design of virtualization and abstract interfaces, the cross-platform performance consistency can be ensured, and the development complexity can be reduced.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to data processing methods. This specification also relates to a data processing system, a computing device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] A self-developed game engine designed based on the hardware memory architecture of a PC can run on a PC platform. However, with the rapid development of heterogeneous computing devices and game engine technology, the demand for cross-device and cross-platform solutions for self-developed game engines is growing.

[0003] Due to the significant differences in hardware memory architecture across different platforms, enabling the engine to adapt to different platforms and achieve cross-platform consistency is a pressing problem that needs to be solved. Summary of the Invention

[0004] In view of this, embodiments of this specification provide a data processing method. This specification also relates to a data processing system, a computing device, a computer-readable storage medium, and a computer program product to address the aforementioned problems in the prior art.

[0005] According to a first aspect of the embodiments of this specification, a data processing method is provided, comprising: In response to an access request carrying a target resource identifier, query the virtual address mapping relationship corresponding to the target resource identifier; If the virtual address mapping relationship is not found, the target resource type is determined according to the target resource identifier, and the target virtual address to be allocated is determined from the logical memory pool corresponding to the target resource type. The logical memory pool is obtained by dividing the virtual address space into resource types. The target memory allocation interface is invoked to load the target resource corresponding to the target resource identifier into the target physical memory, and a mapping relationship between the target physical memory and the allocated target virtual address is established to generate a virtual address handle corresponding to the target virtual address. The target memory allocation interface integrates the memory allocation interfaces of various host platforms. Access the target resource based on the virtual address handle.

[0006] According to a second aspect of the embodiments of this specification, a data processing system applied to a target engine is provided, comprising: The virtual address mapping module is configured to, in response to an access request carrying a target resource identifier, query the virtual address mapping relationship corresponding to the target resource identifier; if the virtual address mapping relationship is not found, determine the target resource type based on the target resource identifier, determine the allocated target virtual address from the logical memory pool corresponding to the target resource type, call the target memory allocation interface, load the target resource corresponding to the target resource identifier into the target physical memory, establish the mapping relationship between the target physical memory and the allocated target virtual address, and generate a virtual address handle corresponding to the target virtual address; The device capability resolution module is configured to obtain the hardware information of the current host platform, wherein the target engine runs on the current host platform; The memory quota allocator module is configured to divide the virtual address space according to the hardware information and engine subsystems of different resource types, determine the logical memory pools corresponding to each resource type of engine subsystem, and allocate virtual memory quotas to each logical memory pool. The virtual memory quotas of each logical memory pool will be dynamically adjusted according to the resource access popularity during the operation of the target engine. The memory fragmentation reclamation module is configured to identify the memory to be reclaimed in the virtual address space, generate a memory fragmentation reclamation task, determine the platform state of the target engine corresponding to the current host platform, and, if the platform state is the target state, reclaim the memory to be reclaimed according to the memory fragmentation reclamation task and release the physical memory corresponding to the memory to be reclaimed.

[0007] According to a third aspect of the embodiments of this specification, a computing device is provided, including a memory, a processor, and a computer program / instructions stored in the memory and executable on the processor, wherein the processor executes the computer program / instructions to implement the steps of the data processing method.

[0008] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores a computer program / instructions that, when executed by a processor, implement the steps of the data processing method.

[0009] According to a fifth aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described data processing method.

[0010] The data processing method provided in this specification responds to resource access requests by querying virtual address mapping relationships, ensuring rapid location and access to loaded resources and effectively reducing access latency. When a mapping is not found, the resource type is determined based on the target resource identifier, and a target virtual address is allocated from the logical memory pool corresponding to the resource type. This ensures that memory resource allocation strictly follows a preset strategy based on resource usage, guaranteeing the orderliness of memory organization. Simultaneously, it allows subsequent physical resource placement to be dynamically associated with the virtual pool category, improving the rationality and targeting of memory configuration. The method calls a target memory allocation interface that integrates the underlying interfaces of various host platforms to load the target resource into physical memory and establish a mapping relationship with the virtual address, ultimately generating a virtual address handle. By abstracting and unifying the differences in physical memory allocation and mapping across different host platforms, the portability and maintainability of the code are greatly improved. The target resource can be accessed through the returned virtual address handle. Through the design of virtualization and abstract interfaces, cross-platform performance consistency is ensured, and development complexity is reduced. Attached Figure Description

[0011] Figure 1 This is a flowchart of a data processing method provided in one embodiment of this specification; Figure 2 This is a schematic diagram illustrating the virtual address mapping process in a data processing method provided in one embodiment of this specification; Figure 3 This is a schematic diagram illustrating the allocation of memory quotas in a data processing method provided in one embodiment of this specification. Figure 4 This is a schematic diagram of memory reclamation in a data processing method provided in one embodiment of this specification; Figure 5 This is a schematic diagram of the structure of a data processing system provided in one embodiment of this specification; Figure 6 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0012] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0013] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0014] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0015] With the rapid development of heterogeneous computing devices and game engine technology, self-developed game engines face higher demands for cross-platform adaptability. However, current self-developed engines designed based on a specific host platform architecture encounter the following systemic technical challenges caused by differences in hardware memory architecture when targeting different host platforms: First, at the memory architecture level, PC platforms generally adopt a dual-pool design with separate CPU (Central Processing Unit) system memory and GPU (Graphics Processing Unit) dedicated video memory, while mainstream host platforms mostly use a unified memory architecture or a single-pool GDDR (Graphics Double Data Rate SDRAM, designed specifically for graphics processing needs and employing double data rate technology) memory. This fundamental difference makes it difficult to directly apply resource placement strategies based on PC architecture to host platforms: on the one hand, the memory bandwidth tiering characteristics commonly found on host platforms (such as separating high-bandwidth and standard-bandwidth regions) require refined bandwidth allocation strategies; on the other hand, the dedicated memory regions reserved for the system by each platform further increase the complexity of memory planning.

[0016] Secondly, regarding I / O mechanisms, while the customized fast I / O path used by the host platform has the advantage of low latency, its limited memory bandwidth resources are prone to bandwidth contention when the CPU and GPU access concurrently. PC-oriented engine designs, typically based on more generous bandwidth budgets, struggle to effectively cope with the bandwidth contention pressures of the host platform.

[0017] Finally, at the memory management level, the unified memory architecture of the host platform causes the CPU and GPU to share the same memory pool, which leads to: First, the lack of a unified cross-platform resource placement mechanism, making it impossible to automatically optimize memory quota allocation based on device characteristics; Second, memory fragmentation generated after long-term operation is difficult to effectively reclaim, and traditional PC platform memory management strategies cannot achieve low-interference defragmentation while ensuring stable frame rates; Third, insufficient memory reclamation efficiency under high load scenarios, which can easily cause system lag or force a degradation in resource quality.

[0018] These technical deficiencies collectively constitute the main obstacle to achieving high-performance cross-platform operation of self-developed game engines, and there is an urgent need to establish a unified memory management solution that can adapt to different memory architectures.

[0019] This specification provides a data processing method, and also relates to a data processing apparatus, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0020] Figure 1 A flowchart of a data processing method according to an embodiment of this specification is shown, which specifically includes the following steps: Step 102: In response to an access request carrying a target resource identifier, query the virtual address mapping relationship corresponding to the target resource identifier.

[0021] Among them, the target resource identifier can be understood as a marker in the game engine used to uniquely identify a specific game resource (such as textures, models, audio, etc.); the virtual address mapping relationship can be understood as the correspondence between the resource identifier and a mapped region in the virtual address space. This mapping relationship records the location of the resource in virtual memory and the physical memory state behind it.

[0022] Specifically, when an engine or application requests access to a resource, it retrieves whether the resource already exists in virtual memory in the unified virtual address mapping layer and obtains its current mapping state.

[0023] In practical applications, within cross-platform game engines, when the rendering subsystem needs to load a character texture, the system does not directly access the file or memory. Instead, it submits an access request with the target resource identifier corresponding to the character texture to the virtual address mapping module. The virtual address mapping module then searches its maintained mapping table for the record corresponding to the target resource identifier to determine whether the character texture has already been loaded.

[0024] By querying the virtual address mapping relationship corresponding to the target resource identifier, a decision-making basis is provided for subsequent memory allocation, resource loading, or online migration, avoiding redundant loading operations. This design allows the upper-layer logic to be free from concern about the specific location of the resource, laying the foundation for cross-platform transparent memory management, dynamic resource scheduling, and access performance optimization.

[0025] In one or more embodiments of this specification, when the virtual address mapping relationship corresponding to the target resource identifier can be found, the virtual address handle can be obtained efficiently, and the target resource can be accessed accordingly. Specific implementation methods are described below: In response to an access request carrying a target resource identifier, after querying the virtual address mapping relationship corresponding to the target resource identifier, the method further includes: If the virtual address mapping relationship is found, the virtual address handle corresponding to the target resource identifier is obtained according to the virtual address mapping relationship, and the target resource is accessed according to the virtual address handle.

[0026] In this specification, a virtual address handle can be understood as an identifier used to uniquely identify a system resource or object. In the embodiments of this specification, the virtual address handle is used to access resources or objects, and it not only includes a virtual address, but can also integrate platform-specific access control information.

[0027] Specifically, when the query confirms that the target resource corresponding to the target resource identifier has established a virtual memory mapping, the virtual address handle corresponding to the target resource identifier is obtained, and the final access to the target resource is completed through the virtual address handle.

[0028] In practical applications, when the rendering thread needs to access the "city night scene texture" resource, the virtual address mapping layer returns a virtual address handle after querying the valid mapping record corresponding to the target resource identifier of the resource. The "city night scene texture" resource can be obtained through this virtual address handle. The virtual address handle can shield the complex details of the underlying physical memory, enabling stable and efficient access to resources.

[0029] The data processing method provided in the embodiments of this specification avoids multiple address translations and permission checks by accessing resources through virtual address handles, significantly improving the reading efficiency of frequently accessed resources. Furthermore, since virtual address handles abstract the differences in the underlying memory architecture, the same set of access logic can maintain stable operation on different platforms, providing a basic guarantee for cross-platform performance consistency.

[0030] In one or more embodiments of this specification, it is determined whether the performance of the initial physical memory corresponding to the target resource matches the access frequency of the target resource. If they do not match, the target resource is migrated to the target physical memory with the corresponding performance level to achieve efficient access. Specific implementation methods are described below: Before accessing the target resource based on the virtual address handle, the method further includes: Determine the resource access popularity of the target resource corresponding to the target resource identifier, and determine the initial physical memory corresponding to the target resource based on the virtual address handle; If the initial physical memory is a persistent storage area and / or the resource access frequency does not match the performance level of the initial physical memory, the target resource is migrated to the target physical memory, and a mapping relationship between the target physical memory and the target virtual address is established. Update the virtual address handle, and access the target resource based on the updated virtual address handle.

[0031] Among them, resource access popularity can be understood as a frequency index of target resource access obtained through statistical analysis; initial physical memory can be understood as the physical storage medium where the target resource currently resides, which can be GPU, system memory or persistent storage area, without limitation; performance level mismatch can be understood as a resource allocation imbalance state, that is, high access popularity resources are located in low speed storage area, or low access popularity resources occupy high performance memory space.

[0032] Specifically, the physical memory storage location is validated before the target resource is accessed, thereby enabling appropriate data migration.

[0033] For example, when multiple characters continuously approach a building, the access frequency of the building's texture resources increases. Before receiving an access request for the texture resource and actually executing the access, the system first detects that although the texture resource has a virtual address mapping, its initial physical memory was swapped to the solid-state drive persistent storage area (a low-speed storage medium) due to previous system memory pressure. At this time, the system recognizes the performance mismatch between the high-access texture resource and the low-speed storage medium, triggering a migration task: loading the texture data completely from the solid-state drive persistent storage area to the GPU's dedicated video memory (i.e., the target physical memory), establishing a new mapping relationship between the target physical memory and the target virtual address, and synchronously updating the virtual address handle to point to the high-speed target physical memory, thereby greatly improving the actual access performance.

[0034] The data processing method provided in the embodiments of this specification significantly optimizes the overall utilization efficiency of memory resources by performing checks before actual access to resources and enabling dynamic migration. This ensures that high-performance physical memory prioritizes resource access requests with higher access frequency, effectively avoiding access delays caused by improper resource residency.

[0035] Step 104: If the virtual address mapping relationship is not found, determine the target resource type based on the target resource identifier, and determine the allocated target virtual address from the logical memory pool corresponding to the target resource type, wherein the logical memory pool is obtained by dividing the virtual address space into resource types.

[0036] Among them, the target resource type can be understood as a classification based on resource characteristics, such as texture, material, and map; the logical memory pool can be understood as a dedicated area pre-divided based on resource type within a unified virtual address space, with each memory pool serving a specific type of engine subsystem.

[0037] Specifically, when the system confirms that the target resource of the access request has not yet been loaded, it resolves the type of the target resource through the target resource identifier, and then allocates a virtual address space for it in the corresponding logical memory pool, namely the target virtual address, to prepare for subsequent physical memory mapping and resource data loading.

[0038] In practice, when a game needs to load a high-definition background texture for the first time, the resource is identified as a "texture" type based on its corresponding target resource identifier. A contiguous range of virtual addresses is then allocated from the "rendering memory pool." It's important to note that during initialization, the logical memory pool is mapped in the virtual address space to an area that the GPU can efficiently access, based on the output of the device capability resolution module. For example, on a host platform with a dedicated graphics card, the rendering memory pool may be preferentially mapped to the virtual address segment corresponding to the video memory, while on a host platform with a unified memory architecture, it may be mapped to a high-bandwidth memory region.

[0039] The data processing method provided in the embodiments of this specification divides the virtual address space into resource types to obtain different logical memory pools. Based on the different logical memory pools, it ensures that resources with different characteristics can be allocated to the physical memory region that best matches their access characteristics, thereby optimizing memory access performance. This can effectively avoid mutual interference between different types of resources and reduce performance bottlenecks caused by improper memory allocation strategies.

[0040] In one or more embodiments of this specification, the hardware information of the current host platform is obtained through the target engine's device capability resolution module, thereby enabling subsequent logical memory pool partitioning based on the current host platform's hardware information. Specific implementation methods are described below: Before determining the target virtual address to be allocated from the logical memory pool corresponding to the target resource type, the method further includes: Determine the current host platform on which the target engine is running, and obtain the hardware information of the current host platform; Based on the hardware information and engine subsystems of different resource types, the virtual address space is divided to determine the logical memory pool corresponding to each resource type engine subsystem, and a virtual memory quota is allocated to each logical memory pool. The virtual memory quota of each logical memory pool will be dynamically adjusted according to the resource access popularity during the operation of the target engine.

[0041] Among them, the device capability analysis module is responsible for collecting and analyzing specific hardware parameters of the platform, and can obtain detailed hardware information such as CPU, GPU, memory specifications, memory capacity, and bandwidth characteristics; the virtual memory quota is the initial address space capacity allocated to each logical memory pool.

[0042] In practice, when the game engine starts running on platform A (i.e., the current host platform), the device capability analysis module comprehensively detects the hardware information of platform A. Based on the hardware information of platform A, the system creates multiple logical memory pools in the virtual address space. For example, a larger virtual memory quota is set for the "rendering memory pool" allocated to the rendering subsystem, and it is preferentially associated with high-speed bandwidth physical memory regions. The "media resource memory pool" is configured according to its low latency requirements, and the virtual memory quota of each logical memory pool can be dynamically adjusted during the operation of the target engine based on the resource access frequency. The allocation of these logical memory pools strictly follows the hardware capabilities of the current host platform to ensure a reasonable match between the memory architecture and hardware characteristics.

[0043] The data processing method provided in the embodiments of this specification divides the virtual address space according to the hardware characteristics of the current host platform, so that subsequent memory allocation operations can make full use of the hardware performance potential, avoid bandwidth bottlenecks caused by unreasonable resource placement, and the virtual memory quota of the logical memory pool can be readjusted according to the actual access mode during operation to ensure that high-demand target resources obtain sufficient memory space.

[0044] In one or more embodiments of this specification, when allocating virtual addresses for target resources within the logical memory pool, a reasonable target virtual address is allocated from the virtual address space to be occupied. Specific implementation methods are described below: The step of determining the target virtual address to be allocated from the logical memory pool corresponding to the target resource type includes: Determine the logical memory pool corresponding to the target resource type, and determine the virtual address space to be occupied in the logical memory pool based on the virtual memory quota and current capacity of the logical memory pool; The target virtual address to be allocated to the target resource is determined from the virtual address space to be occupied.

[0045] The current capacity represents the size of the address space already allocated and used by the logical memory pool; the virtual address space to be occupied is the remaining available address range calculated by the difference between the virtual memory quota and the current capacity.

[0046] Specifically, after determining the target logical memory pool, the virtual address space to be occupied is determined by the virtual memory quota and the current capacity. This virtual address space to be occupied is the remaining available virtual address space, and a specific segment is allocated from the available virtual address space as the target virtual address.

[0047] In practical applications, when the rendering subsystem requests to allocate a virtual address for a texture resource, it determines the "rendering memory pool" corresponding to the texture resource. If the virtual memory quota of the rendering memory pool is 2GB and the current capacity is 1.2GB, then the remaining available address space is 800MB. Within this 800MB address range, the required space is calculated based on the actual size of the texture data, and the corresponding virtual address segment is allocated from the current allocation pointer position of the rendering memory pool.

[0048] The data processing method provided in the embodiments of this specification can effectively isolate and manage virtual memory resources through different logical memory pools, and can prevent a single resource type from excessively occupying the virtual address space and affecting the normal operation of other engine subsystems, thereby improving the overall stability of the system.

[0049] Step 106: Call the target memory allocation interface, load the target resource corresponding to the target resource identifier into the target physical memory, establish the mapping relationship between the target physical memory and the allocated target virtual address, and generate the virtual address handle corresponding to the target virtual address. The target memory allocation interface integrates the memory allocation interfaces of each host platform.

[0050] Among them, the target memory allocation interface encapsulates and unifies the memory allocation functions (i.e., memory allocation interface) specific to each host platform.

[0051] In practical applications, when the system needs to allocate video memory for a character model, it calls the target memory allocation interface. On host platform A, the target memory allocation interface internally calls the corresponding memory allocation function of host platform A to allocate target physical memory in the high-speed video memory area; on host platform B, the target memory allocation interface internally requests target physical memory through the corresponding memory allocation interface of host platform B; after the target physical memory is allocated, the system loads the model data from the storage device into the target physical memory, and at the same time establishes a mapping relationship between the target virtual address in the logical memory pool and the target physical memory, and finally generates a virtual address handle containing mapping information and returns it to the rendering subsystem.

[0052] By implementing a unified target memory allocation interface, cross-platform memory management abstraction is achieved. By encapsulating the memory allocation semantics specific to each host platform, a consistent target memory allocation interface is provided to the upper layer, which significantly reduces the complexity and maintenance cost of multi-platform development. The generated virtual address handle serves as a stable access identifier, ensuring that even if the underlying physical memory changes location due to optimization operations, the upper layer logic can still correctly access the target resource through the virtual address handle.

[0053] In one or more embodiments of this specification, when the target memory allocation interface is called, the memory allocation interface corresponding to the current host platform will be specifically called within the call to allocate the target physical memory. The specific implementation is as follows: The step of calling the target memory allocation interface and loading the target resource corresponding to the target resource identifier into the target physical memory includes: The target memory allocation interface is invoked, and the memory allocation interface corresponding to the current host platform is invoked according to the target memory allocation interface to allocate target physical memory for the target resource and load the target resource corresponding to the target resource identifier into the target physical memory.

[0054] Among them, the memory allocation interface is the native memory management function provided by each console platform. For example, when Xbox is the current console platform, the memory allocation interface is the XMemAlloc memory allocation function. When PS5 or Switch2 is the current console platform, the memory allocation interface is the memory management interface provided by the platform SDK.

[0055] Specifically, when calling the target memory allocation interface, the actual physical memory allocation is completed by identifying the current host platform and the corresponding parameter conversion, and utilizing the memory allocation interface of the current host platform.

[0056] For example, when memory needs to be allocated for resources such as high-resolution textures, the system initiates a request through a unified target memory allocation interface. This interface, as an abstraction layer, can automatically adapt to the underlying memory management mechanisms of different host platforms, achieving cross-platform compatibility.

[0057] On the Xbox platform, the target memory allocation interface is translated into a call to the platform's native XMemAlloc function at the underlying level. The XMemAlloc function is a unified memory allocation interface provided by Xbox, which supports both physical memory allocation and heap allocation modes and has a high degree of customizability. Developers and tools can intercept the XMemAlloc function to implement custom behaviors such as log tracking and memory interception.

[0058] For the PS5 and Switch 2 platforms, since they do not provide a single public memory allocation function like Xbox, but instead adopt a highly customized SDK solution, the target memory allocation interface will be converted into a call to the dedicated memory management interface provided by the platform SDK. These dedicated interfaces include specific functions such as allocating buffers for graphics / audio resources and controlling memory alignment, while combining with standard general memory allocation interfaces (such as malloc / free, new / delete, etc.) to meet the memory allocation needs in different scenarios.

[0059] Once the platform-specific memory allocation interface is executed successfully, it will return the allocated target physical memory. Based on the target physical memory, the loading of resource data will continue, thereby providing a consistent memory access experience for the upper layer.

[0060] The data processing method provided in the embodiments of this specification maintains the stability and consistency of the upper-level logic through the target memory allocation interface, eliminates the complexity caused by platform differences, provides a unified control point for subsequent memory optimization operations, and enables memory management strategies to remain consistent across different platforms by encapsulating platform-specific details within the abstract interface.

[0061] In one or more embodiments of this specification, when the target resource has been mapped, physical memory migration can be achieved based on resource access frequency before actual access. However, when the target resource has not been mapped, it is necessary to allocate physical memory reasonably based on resource access frequency, that is, to match the target performance level of the target physical memory with the resource access frequency. Specific implementation methods are described below: Loading the target resource corresponding to the target resource identifier into the target physical memory includes: Determine the resource access popularity of the target resource corresponding to the target resource identifier, and allocate target physical memory of the target performance level to the target resource according to the resource access popularity, wherein the target physical memory is the physical memory of the target engine corresponding to the current host platform; Load the target resource into the target physical memory of the target performance level; The process of establishing the mapping relationship between the target physical memory and the allocated target virtual address includes: Establish a mapping relationship between the target physical memory and the target virtual address in the logical memory pool, wherein the mapping relationship includes the target performance level corresponding to the target physical memory, and the target performance level is related to the resource access popularity.

[0062] The target performance level is a hierarchy based on the actual bandwidth, latency and other characteristics of physical memory, such as high-speed graphics memory and ordinary system memory; the mapping relationship not only includes the correspondence between virtual addresses and physical addresses, but also records the performance attributes of physical memory.

[0063] Specifically, during the resource loading phase, physical memory of appropriate performance level is allocated to the target resource, and a complete mapping relationship containing performance metadata is established.

[0064] In practical applications, when a game engine needs to load a character skill effect resource, the system analyzes the usage frequency and scene importance of the character skill effect resource in the game to determine that it has high access popularity. The system allocates physical space in the high-bandwidth memory area of ​​the current host platform through the host platform's specific memory allocation interface, rather than the ordinary system memory area. The effect resource is directly loaded into this high-speed memory. At the same time, when establishing the mapping relationship between the target physical memory and the target virtual address in the logical memory pool, the high-performance level attribute of the target physical memory is clearly marked.

[0065] The data processing method provided in this specification effectively avoids the additional overhead caused by subsequent migration by matching the target physical memory of the target performance level according to the access characteristics during the resource loading stage. In addition, when the mapping relationship includes the performance level, it provides a decision basis for subsequent dynamic adjustment, enabling the system to continuously track and verify the rationality of resource placement. This method significantly improves the overall efficiency of the memory system, ensuring that high-performance physical memory prioritizes serving bandwidth-sensitive resource access requests, thereby maintaining stable performance in complex multi-platform environments and providing an underlying guarantee for the smoothness of the game experience.

[0066] Step 108: Access the target resource based on the virtual address handle.

[0067] Applications or engine subsystems can securely and efficiently access target resources through virtual address handles without needing to concern themselves with the underlying implementation details.

[0068] In one or more embodiments of this specification, during the operation of the target engine, unreclaimed memory in the virtual address space can be identified, thereby reclaiming fragmented unreclaimed memory and releasing physical memory, thus significantly reducing intra-frame blocking caused by memory resource contention. Specific implementation methods are described below: The data processing method further includes: Identify the memory to be reclaimed in the virtual address space and generate a memory fragmentation reclamation task; Determine the platform state of the target engine corresponding to the current host platform. If the platform state is the target state, reclaim the memory to be reclaimed according to the memory fragmentation reclamation task, and release the physical memory corresponding to the memory to be reclaimed.

[0069] Among them, the platform state reflects the current runtime environment of the host platform, mainly the load performance indicators of the host platform; the target state can be understood as the platform state suitable for performing memory reclamation operations, which usually corresponds to the idle or low load state.

[0070] Specifically, it continuously monitors memory usage in the virtual address space and triggers memory fragmentation reclamation tasks at appropriate times to optimize memory space continuity.

[0071] In practical applications, when the game scene changes, a large number of scene-specific resources are marked as releasable, resulting in a large number of scattered small free areas in the virtual address space. After detecting this fragmentation, a corresponding memory fragmentation reclamation task is generated.

[0072] The system continuously monitors the current platform status of the host platform. When it detects that the GPU is idle, under low load, or executing a low-priority task, it determines the current platform status of the host platform as the target status. At this time, it initiates a memory fragmentation reclamation task to move the still occupied valid data to a contiguous memory area, merges scattered free blocks, and finally releases the consolidated physical memory resources. By safely migrating, compressing, and merging the memory to be reclaimed with low overhead, it avoids impacting rendering performance.

[0073] The data processing method provided in the embodiments of this specification effectively avoids frame rate fluctuations or rendering interruptions caused by memory defragmentation by precisely selecting idle and low-load periods of the GPU to perform recycling operations, ensuring the smoothness of the game experience. Furthermore, the automated fragmentation merging process significantly improves the continuity of memory space, providing sufficient space for subsequent large-scale resource loading and reducing resource degradation or loading delays caused by insufficient memory.

[0074] The data processing method provided in one embodiment of this specification is applied to a cross-platform unified virtual memory management system. The system architecture consists of a device capability resolution module, a virtual address mapping module, a memory quota allocator module, and a memory fragmentation recycling module, aiming to achieve intelligent adaptation and efficient utilization of heterogeneous memory architectures.

[0075] The device capability analysis module forms the foundation of the entire architecture. Through deep interaction with the operating system and hardware drivers, this module comprehensively collects and analyzes hardware information from different host platforms, including core performance parameters such as CPU architecture, GPU characteristics, memory capacity, and bandwidth levels. This module provides precise data support for the formulation of subsequent memory management strategies, enabling the system to fully understand and utilize the specific memory capabilities of various host platforms, ensuring that the target engine can achieve optimal performance on different platforms, including various host platforms.

[0076] The virtual address mapping module bears the core responsibility for resource allocation and scheduling. This layer abstracts and encapsulates the platform-specific memory allocation mechanisms, providing a consistent memory allocation interface for the target engine at the upper layer. Specifically, on the Xbox platform, the mapping layer calls its unified XMemAlloc function, while on the PS5 and Switch 2 platforms, it adapts a combination of its highly customized SDK interface and general memory allocation functions. This design allows upper-layer logic to achieve efficient resource allocation without needing to concern itself with the differences between the underlying platforms. In fact, this mapping module can dynamically map virtual addresses to physical memory regions of different performance levels based on the resource's access frequency, popularity attributes, and type characteristics, and supports online migration based on access patterns, ensuring that high-frequency resources always reside in the most suitable physical memory unit.

[0077] See Figure 2 , Figure 2 This is a schematic diagram illustrating the virtual address mapping process in a data processing method provided in one embodiment of this specification.

[0078] Specifically, when the engine initiates a virtual address access request for a resource, it first checks whether the resource has been mapped to virtual memory. If so (i.e., the resource has been mapped), it determines whether a migration task needs to be performed. If the physical memory where the resource currently resides (such as high-speed video memory) matches its high access frequency, or if the resource resides in low-speed storage but its access frequency is low, then no migration is required, and the system directly returns the established virtual address handle, through which the engine efficiently accesses the resource. However, when a mapped resource needs to change its physical location, a migration task needs to be performed, entering the steps of migration and mapping to virtual memory.

[0079] If the resource is not yet mapped to virtual memory at the time of query, the step of loading and mapping to virtual memory is executed. The step of loading and mapping to virtual memory is similar to the step of migrating and mapping to virtual memory, both covering the complete process from physical memory allocation, resource loading to establishing the mapping. Specifically, the system first allocates a target virtual address from its corresponding logical memory pool according to the type of resource (such as texture, model), and calls a unified target memory allocation interface that integrates the memory allocation interfaces of various host platforms. Based on the access frequency of the resource, the preset resource category, and the hardware information provided by the device capability resolution module, a target physical memory with an appropriate performance level is allocated for the resource (for example, placing high-frequency textures into the GPU's high-speed video memory), and the resource data is loaded into it, establishing the mapping relationship between the target physical memory and the target virtual address.

[0080] Once the mapping relationship is established, a virtual address handle is returned. This virtual address handle serves as a unique and stable identifier for the resource in the virtual address space and is returned to the engine. All subsequent accesses by the engine can be performed through this virtual address handle, thereby achieving transparency and efficiency in cross-platform memory management.

[0081] The system memory quota allocator module based on platform capabilities uses the hardware information provided by the device capability parsing module as a basis. By quantitatively analyzing indicators such as the bandwidth characteristics, capacity scale, and access latency of the platform memory, it constructs a memory quota scheme that is precisely matched with the hardware capabilities.

[0082] In its implementation, the allocator module divides system memory into multiple logical memory pools based on the functional characteristics and performance requirements of the engine subsystem. The rendering pool manages graphics resources such as textures, materials, and GPU buffers, employing an allocation strategy closely aligned with GPU memory. The script logic pool is dedicated to allocating memory for game logic, artificial intelligence, and script virtual machines. The physics simulation / animation pool is specifically optimized for massive amounts of objects and data structures. The media resource pool ensures the low-latency loading of resources such as audio and video, while other pools manage non-critical data. This pooled architecture, divided by resource type, ensures that all types of resources receive the memory space most suitable for their characteristics.

[0083] After setting a baseline quota for each memory pool based on hardware capabilities during the initialization phase, the allocator module continuously monitors the actual resource usage and access intensity of each subsystem during engine operation. By analyzing this runtime data in real time, the allocator module can intelligently adjust quotas, such as appropriately expanding the quota of the rendering pool when rendering pressure increases, or prioritizing resource supply to the physics simulation pool during physics-intensive phases. This dynamic adjustment mechanism based on access intensity ensures that limited memory resources always flow to the subsystems that need them most, achieving continuous optimization of memory utilization efficiency.

[0084] Through this intelligent quota management strategy, the system not only overcomes the problems of resource waste or insufficiency in the traditional fixed quota mode, but also maintains the memory supply balance between subsystems under different load scenarios, providing a solid resource guarantee for the stable operation and performance of the game engine.

[0085] See Figure 3 , Figure 3 This diagram illustrates a data processing method for allocating memory quotas according to an embodiment of this specification.

[0086] Multiple logical memory pools are pre-divided within the virtual address space, such as Figure 3 The “rendering memory pool”, “script logic memory pool”, etc. shown in the figure, each memory pool serves a specific type of engine subsystem and is allocated an initial virtual memory quota.

[0087] When the engine is running, when a certain engine subsystem (such as the rendering subsystem) needs memory resources, it will initiate a "subsystem request memory allocation" request. The quota allocator will allocate the resource to the specified memory pool according to the resource type in the request. For example, a texture allocation request will be directed to the "render memory pool".

[0088] The system continuously and periodically determines and updates the capacity of each memory pool. Specifically, based on the actual load of the engine during runtime and the access frequency of resources within each resource pool, it can dynamically adjust the capacity quota of each memory pool and update the memory pool. This ensures that limited memory resources can be intelligently and dynamically rebalanced among the various subsystems of the engine according to real-time demand, thereby optimizing memory utilization overall and ensuring the performance stability of the game at different stages of operation.

[0089] The memory fragmentation recycling module is designed with a low-intrusion, cross-platform fragmentation detection and recycling mechanism, which aims to solve the memory fragmentation problem caused by long-term operation in a unified memory architecture.

[0090] This mechanism continuously monitors the memory status during game engine runtime, intelligently scanning for fragmented memory blocks caused by frequent resource creation and release. When it identifies a large number of scattered small blocks of free memory or low-availability memory (i.e., reclaimable memory), the system automatically generates and submits a background memory migration task (i.e., a memory fragmentation reclamation task). These tasks adhere to strict design principles. First, they ensure low overhead by scheduling reclamation operations during GPU idle periods or low-priority task windows to effectively avoid interfering with critical rendering frames and game logic. Second, they guarantee safety by strictly synchronizing all memory migration and merging operations with the GPU / CPU to ensure that data currently in use is not incorrectly reclaimed or overwritten. Finally, they maintain versatility, flexibly adapting to the characteristics of different platforms, regardless of whether the platform supports advanced memory management features such as sparse binding.

[0091] During execution, this module reorganizes scattered free memory blocks into larger, contiguous memory regions through a safe migration, compression, and merging process. This intelligent defragmentation significantly improves the continuity of memory space, providing ample memory for subsequent large-scale resource loading, while reducing performance degradation caused by memory allocation failures. Through this refined memory maintenance mechanism, the system can maintain high memory utilization during long-term operation, providing a solid foundation for the game engine's stable performance and smooth user experience.

[0092] See Figure 4 , Figure 4 This diagram illustrates a memory reclamation process in a data processing method provided in one embodiment of this specification.

[0093] The system continuously monitors memory status during engine operation and periodically scans fragmented memory blocks to identify small blocks of free memory scattered in the virtual address space due to frequent loading and releasing of resources. If memory fragmentation is confirmed, the system accurately identifies those scattered memory blocks that have been released but whose corresponding physical memory has not yet been reclaimed by the system.

[0094] Based on the identification results, a memory fragmentation reclamation task is generated and submitted, and placed into the background task queue. The platform status of the target engine corresponding to the current host platform is determined, that is, whether there is an idle window for the GPU / system. The idle window refers to the GPU command queue being completed or the GPU being in a low-priority computing task cycle.

[0095] If the system is detected to be under heavy load, the reclamation task will be queued to avoid interfering with critical rendering frames or game logic. Once the system confirms the existence of a suitable free window, the reclamation task will be executed immediately. Through safe relocation and merging operations, scattered small blocks of free memory will be organized into larger, contiguous memory blocks. After the task is completed, memory reclamation logs will be compiled and reported to provide developers with data for memory optimization and system tuning.

[0096] By intelligently selecting the timing of memory reclamation at runtime, a balance is achieved between memory defragmentation and game performance, ensuring that the game can maintain efficient memory usage and a stable frame rate even after long-term operation.

[0097] The virtual memory management system provided in this specification achieves dynamic optimization of memory resource allocation through intelligent online migration technology based on memory access popularity and a precise fragmentation recycling mechanism. This mechanism can automatically schedule high-frequency resources to the optimal memory region according to real-time usage scenarios and memory fragmentation distribution, while continuously maintaining the continuity of memory space through background defragmentation. This dynamic management strategy enables the system to achieve higher effective memory utilization on different host platforms under the same physical memory capacity.

[0098] In terms of performance, the implementation of cross-platform virtual address mapping technology, combined with the subsystem's dynamic memory allocation and timely recycling mechanism, effectively solves the memory resource competition problem in multi-platform environments. This comprehensive memory management strategy significantly reduces the intra-frame blocking phenomenon caused by memory allocation conflicts and fragmentation, ensuring that the game engine can maintain a stable frame rate output and a smooth visual experience on different host platforms when processing large-scale data and complex computing tasks.

[0099] In terms of development efficiency, the collaborative design of the unified virtual address mapping module and memory quota allocator module fundamentally simplifies the complexity of multi-platform adaptation. This architecture eliminates the need for the target engine to implement resource placement strategies and numerous platform-related code branches for each specific platform, greatly reducing the engineering complexity of cross-platform development. This not only significantly saves manpower investment in the development and maintenance phases but also greatly reduces the time cost and resource consumption required for multi-platform testing and verification, providing strong infrastructure support for the rapid iteration and efficient deployment of game engine projects.

[0100] Corresponding to the above method embodiments, this specification also provides embodiments of a data processing system applied to a target engine. Figure 5 A schematic diagram of the structure of a data processing system provided in one embodiment of this specification is shown. Figure 5 As shown, the system includes: The virtual address mapping module 502 is configured to, in response to an access request carrying a target resource identifier, query the virtual address mapping relationship corresponding to the target resource identifier; if the virtual address mapping relationship is not found, determine the target resource type based on the target resource identifier, determine the allocated target virtual address from the logical memory pool corresponding to the target resource type, call the target memory allocation interface, load the target resource corresponding to the target resource identifier into the target physical memory, establish the mapping relationship between the target physical memory and the allocated target virtual address, and generate a virtual address handle corresponding to the target virtual address. The target memory allocation interface integrates the memory allocation interfaces of various host platforms. The device capability resolution module 504 is configured to obtain the hardware information of the current host platform, wherein the target engine runs on the current host platform; The memory quota allocator module 506 is configured to divide the virtual address space according to the hardware information and engine subsystems of different resource types, determine the logical memory pools corresponding to the engine subsystems of each resource type, and allocate virtual memory quotas to each logical memory pool. The virtual memory quotas of each logical memory pool will be dynamically adjusted according to the resource access popularity during the operation of the target engine. The memory fragmentation reclamation module 508 is configured to identify the memory to be reclaimed in the virtual address space, generate a memory fragmentation reclamation task, determine the platform state of the target engine corresponding to the current host platform, and, if the platform state is the target state, reclaim the memory to be reclaimed according to the memory fragmentation reclamation task and release the physical memory corresponding to the memory to be reclaimed.

[0101] Optionally, the memory quota allocator module 506 is further configured to: Determine the logical memory pool corresponding to the target resource type, and determine the virtual address space to be occupied in the logical memory pool based on the virtual memory quota and current capacity of the logical memory pool; The target virtual address to be allocated to the target resource is determined from the virtual address space to be occupied.

[0102] Optionally, the virtual address mapping module 502 is further configured to: The target memory allocation interface is invoked, and the memory allocation interface corresponding to the current host platform is invoked according to the target memory allocation interface to allocate target physical memory for the target resource and load the target resource corresponding to the target resource identifier into the target physical memory.

[0103] Optionally, the virtual address mapping module 502 is further configured to: Determine the resource access popularity of the target resource corresponding to the target resource identifier, and allocate target physical memory of the target performance level to the target resource according to the resource access popularity, wherein the target physical memory is the physical memory of the target engine corresponding to the current host platform; Load the target resource into the target physical memory of the target performance level; Establish a mapping relationship between the target physical memory and the target virtual address in the logical memory pool, wherein the mapping relationship includes the target performance level corresponding to the target physical memory, and the target performance level is related to the resource access popularity.

[0104] Optionally, the virtual address mapping module 502 is further configured to: If the virtual address mapping relationship is found, the virtual address handle corresponding to the target resource identifier is obtained according to the virtual address mapping relationship.

[0105] Optionally, the virtual address mapping module 502 is further configured to: Determine the resource access popularity of the target resource corresponding to the target resource identifier, and determine the initial physical memory corresponding to the target resource based on the virtual address handle; If the initial physical memory is a persistent storage area and / or the resource access frequency does not match the performance level of the initial physical memory, the target resource is migrated to the target physical memory, a mapping relationship between the target physical memory and the target virtual address is established, and the virtual address handle is updated.

[0106] The above is an illustrative scheme of a data processing system according to this embodiment. It should be noted that the technical solution of this data processing system and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the data processing system, please refer to the description of the technical solution of the data processing method described above.

[0107] Figure 6 A structural block diagram of a computing device 600 according to an embodiment of this specification is shown. The components of the computing device 600 include, but are not limited to, a memory 610 and a processor 620. The processor 620 is connected to the memory 610 via a bus 630, and a database 650 is used to store data.

[0108] The computing device 600 also includes an access device 640, which enables the computing device 600 to communicate via one or more networks 660. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 640 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0109] In one embodiment of this specification, the above-described components of the computing device 600 and Figure 6 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 6 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0110] The computing device 600 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 600 can also be a mobile or stationary server.

[0111] The processor 620 implements the data processing method when executing the computer program / instructions.

[0112] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the data processing method described above.

[0113] An embodiment of this specification also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the data processing method as described above.

[0114] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the data processing method described above.

[0115] An embodiment of this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0116] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the above method belong to the same concept, and all details not described in detail in the technical solution of the computer program product can be referred to the description of the technical solution of the above method.

[0117] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0118] The computer program / instructions include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0119] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this specification is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this specification. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this specification.

[0120] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0121] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. These embodiments have been selected and specifically described in this specification to better explain the principles and practical applications of this specification, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A data processing method, characterized in that, include: In response to an access request carrying a target resource identifier, query the virtual address mapping relationship corresponding to the target resource identifier; If the virtual address mapping relationship is not found, the target resource type is determined according to the target resource identifier, and the target virtual address to be allocated is determined from the logical memory pool corresponding to the target resource type. The logical memory pool is obtained by dividing the virtual address space into resource types. The target memory allocation interface is invoked to load the target resource corresponding to the target resource identifier into the target physical memory, and a mapping relationship between the target physical memory and the allocated target virtual address is established to generate a virtual address handle corresponding to the target virtual address. The target memory allocation interface integrates the memory allocation interfaces of various host platforms. Access the target resource based on the virtual address handle.

2. The data processing method according to claim 1, characterized in that, Before determining the target virtual address to be allocated from the logical memory pool corresponding to the target resource type, the method further includes: Determine the current host platform on which the target engine is running, and obtain the hardware information of the current host platform; Based on the hardware information and engine subsystems of different resource types, the virtual address space is divided to determine the logical memory pool corresponding to each resource type engine subsystem, and a virtual memory quota is allocated to each logical memory pool. The virtual memory quota of each logical memory pool will be dynamically adjusted according to the resource access popularity during the operation of the target engine.

3. The data processing method according to claim 2, characterized in that, The step of determining the target virtual address to be allocated from the logical memory pool corresponding to the target resource type includes: Determine the logical memory pool corresponding to the target resource type, and determine the virtual address space to be occupied in the logical memory pool based on the virtual memory quota and current capacity of the logical memory pool; The target virtual address to be allocated to the target resource is determined from the virtual address space to be occupied.

4. The data processing method according to claim 3, characterized in that, The step of calling the target memory allocation interface and loading the target resource corresponding to the target resource identifier into the target physical memory includes: The target memory allocation interface is invoked, and the memory allocation interface corresponding to the current host platform is invoked according to the target memory allocation interface to allocate target physical memory for the target resource and load the target resource corresponding to the target resource identifier into the target physical memory.

5. The data processing method according to claim 1 or 4, characterized in that, Loading the target resource corresponding to the target resource identifier into the target physical memory includes: Determine the resource access popularity of the target resource corresponding to the target resource identifier, and allocate target physical memory of the target performance level to the target resource according to the resource access popularity, wherein the target physical memory is the physical memory of the target engine corresponding to the current host platform; Load the target resource into the target physical memory of the target performance level; The process of establishing the mapping relationship between the target physical memory and the allocated target virtual address includes: Establish a mapping relationship between the target physical memory and the target virtual address in the logical memory pool, wherein the mapping relationship includes the target performance level corresponding to the target physical memory, and the target performance level is related to the resource access popularity.

6. The data processing method according to any one of claims 1-4, characterized in that, In response to an access request carrying a target resource identifier, after querying the virtual address mapping relationship corresponding to the target resource identifier, the method further includes: If the virtual address mapping relationship is found, the virtual address handle corresponding to the target resource identifier is obtained according to the virtual address mapping relationship, and the target resource is accessed according to the virtual address handle.

7. The data processing method according to claim 6, characterized in that, Before accessing the target resource based on the virtual address handle, the method further includes: Determine the resource access popularity of the target resource corresponding to the target resource identifier, and determine the initial physical memory corresponding to the target resource based on the virtual address handle; If the initial physical memory is a persistent storage area and / or the resource access frequency does not match the performance level of the initial physical memory, the target resource is migrated to the target physical memory, and a mapping relationship between the target physical memory and the target virtual address is established. Update the virtual address handle, and access the target resource based on the updated virtual address handle.

8. The data processing method according to any one of claims 1-4 and 7, characterized in that, Also includes: Identify the memory to be reclaimed in the virtual address space and generate a memory fragmentation reclamation task; Determine the platform state of the target engine corresponding to the current host platform. If the platform state is the target state, reclaim the memory to be reclaimed according to the memory fragmentation reclamation task, and release the physical memory corresponding to the memory to be reclaimed.

9. A data processing system applied to a target engine, characterized in that, include: The virtual address mapping module is configured to respond to an access request carrying a target resource identifier, query the virtual address mapping relationship corresponding to the target resource identifier, and if the virtual address mapping relationship is not found, determine the target resource type based on the target resource identifier, determine the allocated target virtual address from the logical memory pool corresponding to the target resource type, call the target memory allocation interface, load the target resource corresponding to the target resource identifier into the target physical memory, establish the mapping relationship between the target physical memory and the allocated target virtual address, and generate a virtual address handle corresponding to the target virtual address. The target memory allocation interface integrates the memory allocation interfaces of various host platforms. The device capability resolution module is configured to obtain the hardware information of the current host platform, wherein the target engine runs on the current host platform; The memory quota allocator module is configured to divide the virtual address space according to the hardware information and engine subsystems of different resource types, determine the logical memory pools corresponding to each resource type of engine subsystem, and allocate virtual memory quotas to each logical memory pool. The virtual memory quotas of each logical memory pool will be dynamically adjusted according to the resource access popularity during the operation of the target engine. The memory fragmentation reclamation module is configured to identify the memory to be reclaimed in the virtual address space, generate a memory fragmentation reclamation task, determine the platform state of the target engine corresponding to the current host platform, and, if the platform state is the target state, reclaim the memory to be reclaimed according to the memory fragmentation reclamation task and release the physical memory corresponding to the memory to be reclaimed.

10. A computing device, characterized in that, include: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, wherein when the computer programs / instructions are executed by the processor, they implement the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program / instructions that, when executed by a processor, implement the method described in any one of claims 1 to 8.

12. A computer program product, characterized in that, Includes a computer program / instruction that, when executed by a processor, implements the method according to any one of claims 1 to 8.

Images