Heterogeneous computing method, device and equipment and computer readable storage medium

Abstract

The invention provides a heterogeneous computing method, a heterogeneous computing device, heterogeneous computing equipment and a computer readable storage medium, and is applied to the technical field of data processing. The method comprises the following steps: when a heterogeneous computing power card accesses data, a central processing unit firstly determines whether a target object accessed this time exists in a video memory or not; and under the condition that the target object does not exist in the video memory, the central processing unit reads the target object from the memory and stores the target object in the video memory. Further, the central processing unit can provide the target object loaded into the video memory for the heterogeneous computing power card. In the scheme provided by the embodiment of the invention, the heterogeneous computing data meeting the preset condition is transferred from the video memory to the memory, and the data written into the memory is subjected to a realistic copying strategy, so that the storage space of the heterogeneous computing data can be expanded, and the utilization rate of the video memory space can be improved, thereby realizing video memory oversell, and improving the user experience. And thus, high-performance calculation of the heterogeneous computing power card can be guaranteed.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing technology, and in particular to a heterogeneous computing method, a heterogeneous computing apparatus, a heterogeneous computing device, and a computer-readable storage medium. Background Art

[0002] The heterogeneous computing process primarily involves data exchange between heterogeneous computing cards and video memory. However, video memory has limited space, for example, 40GB or 48GB. This severely restricts the high-performance computing capabilities of heterogeneous computing cards. Therefore, increasing the available video memory space can be a key to breaking through the bottleneck of heterogeneous computing. Summary of the Invention

[0003] The present application provides a heterogeneous computing method, a heterogeneous computing apparatus, a heterogeneous computing device, and a computer-readable storage medium, which can increase the available space of video memory.

[0004] In a first aspect, the present application provides a heterogeneous computing method, which includes: determining whether a target object to be accessed by a heterogeneous computing power card exists in a video memory, wherein the heterogeneous computing data stored in the above-mentioned video memory will be transferred to the internal memory if a preset condition is met; if the above-mentioned target object does not exist in the above-mentioned video memory, determining the above-mentioned target object from the internal memory, and loading the above-mentioned target object from the above-mentioned internal memory into the above-mentioned video memory; and providing the above-mentioned target object in the above-mentioned video memory to the above-mentioned heterogeneous computing power card.

[0005] In one implementation, based on the above solution, the above preset conditions include any one of the following: not being accessed within a first preset time period and the access frequency within a second preset time period being less than a preset threshold.

[0006] In one implementation, based on the above-mentioned scheme, the above-mentioned method also includes: when it is determined that the heterogeneous computing data stored in the above-mentioned video memory meets the above-mentioned preset conditions, it will be transferred to the internal memory, and after the heterogeneous computing data stored in the above-mentioned video memory is transferred to the internal memory, the storage space of the transferred data in the above-mentioned video memory will be set as visible space.

[0007] In one implementation, based on the above-mentioned scheme, the video memory address information and memory address information of the above-mentioned heterogeneous computing data are stored through a logical mapping table; wherein the above-mentioned video memory address information is the current storage address of the above-mentioned heterogeneous computing data in the above-mentioned video memory, and the above-mentioned memory address information is the current storage address of the above-mentioned heterogeneous computing data in the above-mentioned memory.

[0008] In one implementation, based on the above-mentioned scheme, the above-mentioned method also includes: when heterogeneous computing data is transferred from the above-mentioned video memory to the above-mentioned memory, or when heterogeneous computing data stored in the above-mentioned memory is accessed, updating the video memory address information and memory address information in the above-mentioned logical mapping table.

[0009] In one implementation, based on the above-mentioned scheme, the above-mentioned updating of the video memory address information and the memory address information in the above-mentioned logical mapping table includes: if the above-mentioned heterogeneous computing data is stored in the above-mentioned video memory and meets the above-mentioned preset conditions, the above-mentioned heterogeneous computing data is transferred to the memory; if the above-mentioned heterogeneous computing data is not accessed during the transfer process, the transferred video memory address information and the memory address information are updated to the above-mentioned logical mapping table; and if the above-mentioned heterogeneous computing data is accessed during the transfer process, the above-mentioned heterogeneous computing data is kept unchanged in the above-mentioned video memory, and the video memory address information and the memory address information before the transfer in the logical mapping table are kept unchanged, and the length of time that the above-mentioned heterogeneous computing data is not accessed is re-timed.

[0010] In one implementation, based on the above-mentioned scheme, the above-mentioned updating of the video memory address information and the memory address information in the above-mentioned logical mapping table includes: if the above-mentioned heterogeneous computing data is stored in the above-mentioned memory and is accessed, the above-mentioned heterogeneous computing data is transferred from the above-mentioned memory to the above-mentioned video memory, and the transferred video memory address information and the memory address information are updated to the above-mentioned logical mapping table.

[0011] In one implementation, based on the above solution, the check code of the heterogeneous computing data transferred from the video memory to the internal memory is further stored in the above logical mapping table; the above transferring the above heterogeneous computing data from the above internal memory to the above video memory includes: transferring the above heterogeneous computing data from the above internal memory to the above video memory via a bus;

[0012] The above method also includes: verifying the mechanism computing data transferred to the above video memory based on the verification code of the above transferred heterogeneous computing data; wherein, if the verification passes, the above heterogeneous computing data in the above video memory is provided to the above heterogeneous computing power card.

[0013] In one implementation, based on the aforementioned scheme, the above-mentioned determination of whether the target object to be accessed by the heterogeneous computing power card exists in the video memory includes: determining the storage location of the above-mentioned heterogeneous computing data in the above-mentioned video memory based on the video memory address information stored in the above-mentioned logical mapping table; and determining whether the target object to be accessed by the above-mentioned heterogeneous computing power card exists in the above-mentioned video memory based on the above-mentioned determination of the storage location of the above-mentioned heterogeneous computing data in the above-mentioned video memory.

[0014] In one implementation, based on the above-mentioned scheme, the above-mentioned determination of the target object from the above-mentioned memory includes: determining the storage location of the heterogeneous computing data in the above-mentioned memory according to the memory address information stored in the above-mentioned logical mapping table; and, based on the above-mentioned storage location of the heterogeneous computing data in the above-mentioned memory, determining the above-mentioned target object from the above-mentioned memory; wherein the above-mentioned memory address information includes at least one pair of address pointer values, and each pair of the above-mentioned address pointer values ​​includes a memory address start pointer value and a memory address end pointer value.

[0015] In one implementation, based on the aforementioned solution, the above-mentioned memory adopts a shared memory management mode; the space of the above-mentioned memory is related to one or more of the following information: the maximum video memory space size provided to the above-mentioned heterogeneous computing power card, the active video memory ratio and the adjustment factor; among them, the active video memory ratio is the space occupied by active data in the above-mentioned heterogeneous computing data, and the ratio of the above-mentioned maximum video memory space.

[0016] In a second aspect, a heterogeneous computing device is provided, which includes: a first determination module, a second determination module and a control module; wherein the first determination module is used to determine whether a target object to be accessed by a heterogeneous computing power card exists in a video memory, wherein the heterogeneous computing process at least includes data exchange between the heterogeneous computing power card and the video memory, and the heterogeneous computing data stored in the video memory will be transferred to the memory if preset conditions are met; the second determination module is used to determine the target object from the memory when the target object does not exist in the video memory, and load the target object from the memory into the video memory; and the control module is used to provide the target object in the video memory to the heterogeneous computing power card.

[0017] In an exemplary embodiment, based on the above solution, the above preset conditions include any one of the following: no access within a first preset time period and the access frequency within a second preset time period is less than a preset threshold.

[0018] In an exemplary embodiment, based on the above scheme, the above control module is also used to: when it is determined that the heterogeneous computing data stored in the above video memory meets the above preset conditions, it will be transferred to the internal memory; and after the heterogeneous computing data stored in the above video memory is transferred to the internal memory, the storage space of the transferred data in the above video memory will be set as visible space.

[0019] In an exemplary embodiment, based on the above scheme, the video memory address information and memory address information of the above heterogeneous computing data are stored through a logical mapping table; wherein the above video memory address information is the current storage address of the above heterogeneous computing data in the above video memory, and the above memory address information is the current storage address of the above heterogeneous computing data in the above memory.

[0020] In an exemplary embodiment, based on the above scheme, the above heterogeneous computing device also includes: an update module; wherein the above update module is used to: update the video memory address information and memory address information in the above logical mapping table when the heterogeneous computing data is transferred from the above video memory to the above memory, or when the heterogeneous computing data stored in the above memory is accessed.

[0021] In an exemplary embodiment, based on the above scheme, the above update module is specifically used to: if the above heterogeneous computing data is stored in the above video memory and meets the above preset conditions, then the above heterogeneous computing data is transferred to the internal memory; if the above heterogeneous computing data is not accessed during the transfer process, then the video memory address information and the internal memory address information after the transfer are updated to the above logical mapping table; the above update module is also used to: if the above heterogeneous computing data is accessed during the transfer process, then the above heterogeneous computing data is kept unchanged in the above video memory, and the video memory address information and the internal memory address information before the transfer in the logical mapping table are kept unchanged; the above heterogeneous computing device also includes: a timing module; wherein the above timing module is used to, when the heterogeneous computing data is accessed during the transfer process, re-time the length of time that the above heterogeneous computing data is not accessed.

[0022] In an exemplary embodiment, based on the above scheme, if the above heterogeneous computing data is stored in the above memory and is accessed, the above control module is also specifically used to transfer the above heterogeneous computing data from the above memory to the above video memory, and the above update module is also specifically used to update the transferred video memory address information and memory address information to the above logical mapping table.

[0023] In an exemplary embodiment, based on the above solution, the check code of the heterogeneous computing data transferred from the video memory to the internal memory is also saved through the above logical mapping table;

[0024] The control module specifically includes: transferring the heterogeneous computing data from the memory to the video memory via the bus;

[0025] The above-mentioned heterogeneous computing device also includes: a verification module; wherein the above-mentioned verification module is used to: verify the mechanical computing data transferred to the above-mentioned video memory based on the verification code of the above-mentioned transferred heterogeneous computing data; wherein, if the verification passes, the above-mentioned heterogeneous computing data in the above-mentioned video memory is provided to the above-mentioned heterogeneous computing power card.

[0026] In an exemplary embodiment, based on the above scheme, the above-mentioned first determination module is specifically used to: determine the storage location of the above-mentioned heterogeneous computing data in the above-mentioned video memory according to the video memory address information stored in the above-mentioned logical mapping table; and, based on the above-mentioned storage location of the above-mentioned heterogeneous computing data in the above-mentioned video memory, determine whether the target object to be accessed by the above-mentioned heterogeneous computing power card exists in the above-mentioned video memory.

[0027] In an exemplary embodiment, based on the above scheme, the above-mentioned second determination module is specifically used to: determine the storage location of the heterogeneous computing data in the above-mentioned memory according to the memory address information stored in the above-mentioned logical mapping table; and, based on the above-mentioned storage location of the above-mentioned heterogeneous computing data in the above-mentioned memory, determine the above-mentioned target object from the above-mentioned memory; wherein the above-mentioned memory address information includes at least one pair of address pointer values, and each pair of the above-mentioned address pointer values ​​includes a memory address start pointer value and a memory address end pointer value.

[0028] In an exemplary embodiment, based on the above scheme, the above memory adopts a shared memory management mode; the space of the above memory is related to one or more of the following information: the maximum video memory space size provided to the above heterogeneous computing power card, the active video memory ratio and the adjustment factor; wherein, the active video memory ratio is the space occupied by active data in the above heterogeneous computing data, and the above maximum video memory space ratio.

[0029] According to a third aspect, a heterogeneous computing device is provided, which includes: a video memory, a memory, a heterogeneous computing power card, and a central processing unit; wherein the video memory is used to store heterogeneous computing data; the memory is used to store heterogeneous computing data transferred from the video memory; the heterogeneous computing power card is used to exchange data with the video memory to perform heterogeneous computing; and the central processing unit is used to receive an access request from the heterogeneous computing power card, determine the target object from the memory when it is determined that the target object to be accessed does not exist in the video memory, load the target object from the memory into the video memory, and provide the target object in the video memory to the heterogeneous computing power card.

[0030] In a fourth aspect, an electronic device is provided, comprising a processor and a memory; the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the heterogeneous computing method provided by the above-mentioned first aspect and its various implementation methods.

[0031] In a fifth aspect, a chip is provided, comprising: a processor for calling and running a computer program from a memory, so that a device equipped with the chip executes the heterogeneous computing method provided in the first aspect and its various implementations.

[0032] In a sixth aspect, a computer-readable storage medium is provided for storing a computer program, wherein the computer program enables a computer to execute the heterogeneous computing method provided by the first aspect and its various implementations.

[0033] In a seventh aspect, a computer program product is provided, comprising computer program instructions, wherein the computer program instructions enable a computer to execute the heterogeneous computing method provided by the first aspect and its various implementations.

[0034] In an eighth aspect, a computer program is provided, which, when executed on a computer, enables the computer to execute the heterogeneous computing method provided by the first aspect and its various implementations.

[0035] In summary, in the solution provided by the embodiment of the present application, the heterogeneous computing processing flow at least includes data exchange between the heterogeneous computing power card and the video memory. In order to reduce the occupation of the video memory space, the heterogeneous computing data stored in the video memory will be transferred to the internal memory when the preset conditions are met. Therefore, when the heterogeneous computing power card accesses a certain data, the central processing unit first determines whether the target object of this access exists in the video memory. If the target object does not exist in the video memory, the central processing unit determines the target object from the internal memory and loads it from the memory into the video memory. Furthermore, the central processing unit can provide the above-mentioned target object loaded into the video memory to the above-mentioned heterogeneous computing power card. In the solution provided by the embodiment of the present application, the heterogeneous computing data that meets the preset conditions is transferred from the video memory to the internal memory, and the write-to-copy strategy is executed on the data written to the internal memory, which can expand the storage space of the heterogeneous computing data and improve the utilization rate of the video memory space, thereby achieving video memory overselling, which is conducive to ensuring the high-performance computing of the heterogeneous computing power card. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0037] Figure 1A A schematic diagram of a heterogeneous computing power card acquiring a target object in the background technology to which embodiments of the present application can be applied;

[0038] Figure 1B A schematic diagram of a heterogeneous computing card acquiring a target object provided in an embodiment of the present application;

[0039] Figure 2 A schematic diagram of a flow chart of a heterogeneous computing method provided in an embodiment of the present application;

[0040] Figure 3 A schematic diagram of information interaction for creating and updating a logical mapping table provided in an embodiment of the present application;

[0041] Figure 4 A schematic diagram of the structure of the logical mapping table provided in an embodiment of the present application;

[0042] Figure 5 A schematic diagram of information interaction for determining a target object to be accessed and updating a logical mapping table provided in an embodiment of the present application;

[0043] Figure 6 A schematic diagram of a process for executing a training task using heterogeneous computing provided in an embodiment of the present application;

[0044] Figure 7 A schematic diagram of processing heterogeneous computing tasks provided in an embodiment of the present application;

[0045] Figure 8 A schematic diagram of the structure of a heterogeneous computing device provided in an embodiment of the present application;

[0046] Figure 9 A schematic diagram of the structure of a heterogeneous computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0048] It should be noted that the terms "first," "second," and the like in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the numbers used in this manner are interchangeable where appropriate so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. In the embodiments of the present application, "B corresponding to A" means that B is associated with A. In one implementation, B can be determined based on A. However, it should also be understood that determining B based on A does not mean determining B solely based on A, but that B can also be determined based on A and / or other information. In addition, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to these processes, methods, products, or devices. In the description of this application, unless otherwise specified, "plurality" refers to two or more than two.

[0049] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0050] Cloud technology refers to a hosting technology that unifies hardware, software, network and other resources within a wide area network or local area network to achieve data computing, storage, processing and sharing.

[0051] Cloud technology is a general term for network technologies, information technologies, integration technologies, management platform technologies, and application technologies based on the cloud computing business model. It can form a resource pool that can be used flexibly and conveniently on demand. Cloud computing technology will become a crucial support. Backend services for technical network systems, such as video websites, image websites, and more portals, require extensive computing and storage resources. With the rapid development and application of the internet industry, every item will likely have its own unique identifier, requiring transmission to backend systems for logical processing. Different levels of data will be processed separately, and data from various industries will require a strong system backend, which can only be achieved through cloud computing.

[0052] Heterogeneous computing is widely used in artificial intelligence (AI) tasks, graphics processing and rendering tasks, audio and video encoding and decoding tasks, financial computing tasks, etc. This application embodiment uses AI large model tasks as an example for illustration.

[0053] With the increasing development of large AI models, the scale of heterogeneous computing power is also increasing. As mentioned above, due to the limited space of graphics cards, the computing performance of heterogeneous computing power cards is limited. To address this problem, a related technology provides a solution in which the training business party evaluates the amount of video memory required by the model before performing training. When the amount of video memory required is greater than the amount of video memory resources that can be provided by the heterogeneous device, the business party performs optimization work on the use of video memory by the model, or reduces the single training data of the model training to achieve the loading operation to complete the model training. The solution provided by the above-mentioned related technology restricts the computing efficiency of large AI models from the perspective of infrastructure, and also affects the output of high-quality models. In another related technology, the number of heterogeneous devices is expanded to train the boxed model, but the storage cost investment is high. Exemplary reference Figure 1AIn the solutions provided by the related technologies, all heterogeneous computing data is stored in the physical video memory. For example, in the first related technology mentioned above, the amount of adapted data can be determined based on the existing video memory resources and stored in the existing physical video memory. For another example, in the second related technology, the physical video memory of other heterogeneous devices is combined to store the training data of the training task. Figure 1A In the related art, in S1, the heterogeneous computing power card sends an access request for the target object to the central processing unit. In S2, the central processing unit controls the loading of the target object stored in the physical video memory to the heterogeneous computing power card to execute the heterogeneous computing task.

[0054] The embodiments of the present application can solve the above technical problems existing in the related art. Figure 1B , the embodiment of the present application transfers heterogeneous computing data that meets preset conditions from the video memory to the internal memory (S0); when the heterogeneous computing power card sends an access request for the target object to the central processing unit (S1), the central processing unit executes a write-to-copy strategy for the data written to the internal memory (S2 and S3). It can be seen that the solution provided by the embodiment of the present application can expand the storage space of heterogeneous computing data and improve the utilization rate of the video memory space, thereby achieving video memory overselling, which is conducive to ensuring the high-performance computing of the heterogeneous computing power card. Compared with the solutions provided by the above-mentioned related technologies, 1) the embodiments of the present application can increase the amount of available space in the video memory, which can solve the problem of model training that cannot be loaded due to insufficient video memory, and is conducive to accelerating the efficiency of model training business access; 2) it can avoid the operation of adapting and reducing the single training data on the business side due to the limitation of the size of the video memory space, thereby helping to improve the computing efficiency of model training; 3) in the related technologies, since there is no need to adapt the video memory to reduce the amount of single training data, the model training design on the business side is decoupled. The embodiments of the present application do not need to adapt the video memory to reduce the amount of single training data, which is conducive to high-quality output of model training; 4) the solution provided by the embodiments of the present application can avoid the expansion of heterogeneous devices, and intuitively reduce the cost investment of heterogeneous computing devices.

[0055] It should be noted that in the embodiments of the present application, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal can be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected through wired or wireless communication, and this application does not limit this.

[0056] The following describes the technical solutions of the embodiments of the present application in detail through some embodiments. The following embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0057] Figure 2 This is a flow chart of a heterogeneous computing method P200 provided in an embodiment of the present application. The execution subject of the method P200 may be a central processing unit (CPU) in a heterogeneous computing device. Figure 2 , method P200 includes S210 to S230.

[0058] In S210 , it is determined whether the target object to be accessed by the heterogeneous computing power card exists in the video memory, wherein the heterogeneous computing data stored in the video memory will be transferred to the internal memory if a preset condition is met.

[0059] In an exemplary embodiment, the preset conditions include any one of the following: not being accessed within a first preset time period, and being accessed less than a preset threshold within a second preset time period. This indicates that the present embodiment transfers heterogeneous computing data that has not been accessed within a short period of time or has a low access frequency for the current task or current computing power container to the memory, specifically determining the data to be transferred to the memory. This effectively adapts to the personalized needs of the current heterogeneous computing task or current computing power container, while also reducing the data's occupation of video memory and improving the utilization of video memory.

[0060] In an exemplary embodiment, after a heterogeneous computing task is created, all related heterogeneous computing data is initially stored in the video memory. However, during the heterogeneous computing process of this task, the video memory is checked to see whether any of the heterogeneous computing data stored in the video memory meets a preset condition. If so, the heterogeneous computing data meeting the preset condition is transferred to the main memory. Exemplarily, the video memory space corresponding to the transferred data is set as visible space, making it available for storing other data. For example, the video memory requirement of heterogeneous computing task A is 20G, and the video memory capacity of the heterogeneous computing device is 40G. The video memory requirement can be provided for heterogeneous computing task A through the video memory, that is, the 20G data of the heterogeneous computing task can be stored through the video memory. During the execution of heterogeneous computing task A, the CPU determines whether there is data that meets the preset conditions in the above 20G heterogeneous computing data. If so, the data that meets the preset conditions is transferred to the memory, and the video memory space corresponding to the transferred data is set as visible space, so that it can be used to store other data. For example, during the execution of task A, the amount of data stored in the video memory is between 8G and 10G. In other words, the video memory space actually provided to task A by the video memory is less than its actual requirement. It can be seen that the embodiment of the present application can effectively improve the utilization rate of video memory space.

[0061] In an exemplary embodiment, when heterogeneous computing data that meets preset conditions is transferred from video memory to internal memory, the method may be bus transmission; or direct memory access (DMA) may be used to transfer data from video memory to internal memory to improve data transmission efficiency. In subsequent embodiments, when heterogeneous computing data stored in internal memory is accessed, the data may also be transferred from internal memory to video memory via bus transmission; or DMA may be used to transfer data from internal memory to video memory to improve data transmission efficiency, thereby facilitating heterogeneous computing efficiency.

[0062] In S220, if the target object does not exist in the video memory, the target object is determined from the memory and loaded from the memory into the video memory. And, in S230, the target object in the video memory is provided to the heterogeneous computing power card.

[0063] It is understandable that if the CPU determines that the target object to be accessed by the heterogeneous computing power card is stored in the video memory, it can directly provide the target object stored in the video memory to the heterogeneous computing power card. In this case, data exchange between the heterogeneous computing power card and the video memory can proceed normally, and the computing performance of the heterogeneous computing power card can be guaranteed.

[0064] In an exemplary embodiment, if the CPU determines that the target object to be accessed by the heterogeneous computing power card is not stored in the video memory, one possible scenario is that the target object meets the aforementioned preset conditions and has already been transferred to the memory. In this case, the target object in the memory can be first loaded into the video memory, and then the target object in the video memory can be provided to the heterogeneous computing power card. In this case, the target object that meets the preset conditions is transferred to the memory, thereby achieving the technical effect of improving video memory utilization. The copy-on-write strategy is implemented for the target object in the memory, thereby ensuring that data exchange between the heterogeneous computing power card and the video memory can proceed normally, thereby ensuring the computing performance of the heterogeneous computing power card.

[0065] In an exemplary embodiment, if the CPU determines that the target object to be accessed by the heterogeneous computing power card is not stored in the video memory, and another possible scenario is that the target object meets the aforementioned pre-set conditions and is in the process of being transferred to the memory, the CPU can terminate the transfer process, control the target object to remain in the video memory, and provide the target object in the video memory to the heterogeneous computing power card. In this case, data exchange between the heterogeneous computing power card and the video memory can proceed normally, and the computing performance of the heterogeneous computing power card can be guaranteed.

[0066] Figure 2In the solution provided by the present application in the illustrated embodiment, the heterogeneous computing processing flow includes at least data exchange between the heterogeneous computing power card and the video memory. In order to reduce the occupation of the video memory space, the heterogeneous computing data stored in the video memory will be transferred to the internal memory when the preset conditions are met. Therefore, when the heterogeneous computing power card accesses a certain data, the central processing unit first determines whether the target object of the access exists in the video memory. If the target object does not exist in the video memory, the central processing unit determines the target object from the internal memory and loads it from the internal memory into the video memory. Furthermore, the central processing unit can provide the above-mentioned target object loaded into the video memory to the above-mentioned heterogeneous computing power card. In the solution provided by the embodiment of the present application, the heterogeneous computing data that meets the preset conditions is transferred from the video memory to the internal memory, and the write-to-copy strategy is executed on the data written to the internal memory. This can expand the storage space of the heterogeneous computing data and improve the utilization rate of the video memory space, thereby achieving video memory overselling, which is conducive to ensuring the high-performance computing of the heterogeneous computing power card.

[0067] In order to ensure that the CPU can quickly and accurately obtain the above-mentioned target object from the video memory or the internal memory, the embodiment of the present application stores the video memory address information and the memory address information of the above-mentioned heterogeneous computing data through a logical mapping table. The video memory address information is the current storage address of the heterogeneous computing data in the video memory, so that the CPU can efficiently locate the heterogeneous computing data of the current task stored in the video memory according to the video memory address information in the logical mapping table. The memory address information is the current storage address of the heterogeneous computing data in the internal memory, so that the CPU can efficiently locate the heterogeneous computing data of the current task stored in the internal memory according to the memory address information in the logical mapping table.

[0068] In an exemplary embodiment, Figure 3 A schematic diagram of information interaction for creating and updating a logical mapping table provided in an embodiment of the present application.

[0069] refer to Figure 3 ,In S31, the heterogeneous computing power card sends a memory request to the ,central processing unit.

[0070] Exemplary references Figure 4 After creating a heterogeneous computing task, it can be translated into production requirements for multiple computing power containers in S11. The heterogeneous computing power card can send the memory request (S12) for each computing power container to the CPU, where the memory request protocol information is shown in Table 1.

[0071] Table 1

[0072] Task name or task ID Container pod name Required video memory

[0073] After the CPU receives the first video memory request of the heterogeneous computing task, it executes S32' and S32. In S32', the CPU controls the heterogeneous computing data corresponding to the received video memory request to be stored in the video memory. Figure 4 For the video memory request issued by the computing power container 3, the CPU controls the heterogeneous computing data corresponding to the video memory request issued by the computing power container 3 to be stored in the video memory.

[0074] In S32, a logical mapping table is created.

[0075] Exemplary references Figure 4 The logical mapping table contains multiple sets of address information, each of which can correspond to a computing power container. For example, a set of address information containing video memory address information A and memory address information B is used to record the storage address of the heterogeneous computing data of computing power container 3. Figure 3 The above-mentioned video memory address information A can be the physical video memory address addr30-address addr40, and the above-mentioned video memory address information B can be the shared memory pointer address ptr10-memory pointer address ptr20.

[0076] Illustratively, a set of address information included in the logical mapping table may include the content shown in Table 2.

[0077] Table 2

[0078] Field Name describe 1 Task Name The name of the training task, identifying the training process 2 Container pod name The name of each container in the task, identifying the smallest logical unit of resources 3 Video memory requirement size The amount of video memory required in the container to perform training calculations 4 Physical video memory size The size of the allocated physical video memory space 5 Physical video memory address start value The initial address value of the physical space video memory 6 Physical memory address end value The end address value of the physical space video memory 7 Memory size The size of the allocated memory space 8 Memory address start pointer value The initial address value of the memory space 9 Memory address end pointer value The end address value of the memory space 10 Memory data check code When copying data between memory and video memory, the checksum of the data

[0079] Refer to Table 2, where the task name, container pod name, and memory requirement size in rows 1-3 of Table 2 can be determined based on the memory request. The physical memory size, physical memory address start value, and physical memory address end value in rows 4-6 of Table 2 are used to record the storage address of the heterogeneous computing data corresponding to the current container pod in the memory. For example, Figure 4 The memory address information A in the display. The field information memory size, memory address start pointer value, and memory address end pointer value in rows 7-9 of Table 2 are used to record the storage address of the heterogeneous computing data corresponding to the current container pod in the memory. For example, Figure 4 The memory address information B. The field information of row 10 in Table 2 is the memory data check code, which is used to record the check code of the data when copying between the memory data and the video memory data.

[0080] In S33 , the CPU determines whether there is heterogeneous computing data that meets a preset condition in the video memory.

[0081] Exemplarily, when the above preset condition is that the data has not been accessed within the first preset time period, the data stored in the video memory is timed to be unaccessed. If a certain data has not been accessed within the first preset time period, it will be transferred to the memory.

[0082] When it is determined that the heterogeneous computing data meeting the preset condition does not exist in the video memory, S33 is executed again to continuously detect the heterogeneous computing data stored in the video memory.

[0083] When it is determined that there is heterogeneous computing data that meets the preset conditions in the video memory, in order to improve the utilization of the video memory space, S34 and S35 are executed to control the heterogeneous computing data that meets the preset conditions to be transferred from the video memory to the internal memory. Figure 4 In S13, if the heterogeneous computing data stored in the video memory meets the preset conditions, the data will be sunk to the main memory. Taking the data stored at address addr30 to address addr40 in the video memory as an example, if data A meets the preset conditions and there is space for storing data A between the pointer addresses ptr10 and ptr20 in the shared memory, it can be sunk to the shared memory between the pointer addresses ptr10 and ptr20.

[0084] In an exemplary embodiment, since the storage address of heterogeneous computing data changes after the video memory data is transferred to the internal memory, in order to efficiently and accurately determine the storage location of the heterogeneous computing data, after executing S34 and S35, the CPU also executes S36: updating the logical mapping table and setting the corresponding space in the video memory as visible space. This updated logical mapping table can more quickly and accurately provide the latest video memory address information and internal memory address information, facilitating the smooth execution of heterogeneous computing tasks. This embodiment also controls the transfer of heterogeneous computing data that meets preset conditions from the video memory to the internal memory and sets the corresponding space in the video memory as visible space, effectively improving video memory space utilization.

[0085] In an exemplary embodiment, Figure 5 A schematic diagram of information interaction for determining a target object to be accessed and updating a logical mapping table provided in an embodiment of the present application.

[0086] refer to Figure 5 ,In S51, the heterogeneous computing power card sends an access request about the target object to the central processor.

[0087] Exemplary references Figure 4 For the multiple computing power containers obtained by translating the heterogeneous computing tasks, the heterogeneous computing power card can send an access request (S21) of each computing power container to the CPU.

[0088] In S52, the CPU determines whether the target object exists in the video memory.

[0089] For example, the central processing unit can efficiently locate the heterogeneous computing data of the current computing power container stored in the graphics memory based on the graphics memory address information recorded in the above logical mapping table, and then determine whether the above target object exists in the heterogeneous computing data of the current computing power container stored in the graphics memory.

[0090] The central processing unit executes different target object acquisition methods according to the judgment result of S52.

[0091] If the CPU determines that the target object is stored in the graphics memory, S541 and S542 are executed. In S541, the CPU determines that the target object is in the graphics memory. In S542, the CPU controls the loading of the target object from the graphics memory to the heterogeneous computing card. This ensures the smooth execution of heterogeneous computing tasks.

[0092] If the CPU determines that the target object is not stored in the video memory, there are two situations. One situation is that the target object has been transferred to the memory, in which case S531 to S534 are executed. The CPU can efficiently locate the heterogeneous computing data of the current computing power container stored in the memory based on the memory address information recorded in the above-mentioned logical mapping table. Then, it is determined whether the above-mentioned target object exists in the heterogeneous computing data of the current computing power container stored in the memory. The other situation is that the target object is in the process of being transferred from the video memory to the memory, in which case S531' to S533' are executed.

[0093] In S531, the CPU determines that the target object is in the memory. In S532, the CPU controls the target object to be loaded from the memory to the video memory. In S533, the CPU controls the target object to be loaded from the video memory to the heterogeneous computing power card. This ensures the smooth execution of heterogeneous computing tasks. Figure 4 When heterogeneous computing data stored in memory is accessed by a heterogeneous computing power card, the central processing unit performs a write-time copy of the heterogeneous computing data stored in memory. If the target object B to be accessed by the heterogeneous computing power card is stored at address ptr10-addr20 in memory, the central processing unit controls the loading of target object B into physical video memory. If there is space for storing target object B between address addr30 and address addr40 in physical video memory, target object B can be loaded into the physical video memory between address addr30 and address addr40. Furthermore, target object B is controlled to be loaded into the heterogeneous computing power card and provided to the corresponding computing power container, enabling the execution of heterogeneous computing tasks within the computing power container.

[0094] In this exemplary embodiment, since the storage address of heterogeneous computing data in memory changes after being loaded into the video memory, in order to efficiently and accurately determine the storage location of the heterogeneous computing data, after executing S532, the CPU also executes S534: updating the logical mapping table. This updated logical mapping table can more quickly and accurately provide the latest video memory address information and internal memory address information, facilitating the smooth execution of heterogeneous computing tasks.

[0095] In S531', it is determined that the target object is in the process of being transferred from the video memory to the memory. The target object being in the process of being transferred from the video memory to the memory may be a case where the target object has not yet arrived in the memory, or a case where the target object has arrived in the memory but the logical mapping table has not been updated (such as the update of the logical mapping table in the case of S36). Figure 4 If the heterogeneous computing data C stored at address addr30 to address addr40 in the video memory meets the preset conditions, the heterogeneous computing data C should be transferred to the main memory. If the heterogeneous computing data C has not yet been stored in the main memory, or if the heterogeneous computing data C has been transferred to the main memory but the logical mapping table has not yet been updated, it can be considered that the heterogeneous computing data C is in the process of being transferred from the video memory to the main memory.

[0096] In S532', the target object is controlled to continue to be stored in the video memory. And, in S533', the CPU controls the target object to be loaded from the video memory to the heterogeneous computing card. This ensures the smooth execution of heterogeneous computing tasks.

[0097] Since the solution provided by S531 ′-S533 ′ does not involve updating of heterogeneous computing data addresses, there is no need to update the logic mapping table in this case.

[0098] Exemplarily, when the above-mentioned preset condition is that the data has not been accessed within the first preset time period, for the solution provided by S531'-S533', since the target object has been accessed, the target object needs to be re-timed as not being accessed to ensure the accuracy of the data being sunk from the video memory to the internal memory.

[0099] Exemplarily, for the solution provided by S531'-S533', since the target object is transmitted through the bus when it is sunk from the video memory to the internal memory. When the target object in the memory is accessed, it can be loaded from the internal memory to the video memory through the bus. Since the target object is transmitted through the bus, in order to avoid data errors, the embodiment of the present application also verifies the heterogeneous computing data loaded into the video memory according to the verification code in the logical mapping table. Exemplarily, only when the verification is passed will the target object in the video memory be provided to the heterogeneous computing power card, thereby ensuring the accuracy of the data during the heterogeneous computing process.

[0100] Figure 5 In the method provided by the illustrated embodiment, the control operation interface is transformed from directly operating the physical video memory to operating the logical mapping table through a logical mapping table. Based on the video memory address information and the internal memory address information in the logical mapping table, the acquisition operation of the target object to be accessed is implemented. If the target object stored in the video memory meets the preset conditions, it is sunk to the internal memory and the above-mentioned logical mapping table is updated. Therefore, when the target object in the memory is accessed, a copy-on-write strategy is adopted to achieve overselling and reuse of the video memory, thereby improving the utilization rate of the video memory space.

[0101] In an exemplary embodiment, if the target object to be accessed does not exist in the video memory and the internal memory, and is not in the process of being transferred from the video memory to the internal memory, it may indicate an access error, and the central processing unit may issue an abnormal prompt.

[0102] In an exemplary embodiment, the embodiment of the present application improves the utilization rate of video memory space by transferring heterogeneous computing data that meets preset conditions to the memory. The memory that stores the transferred heterogeneous computing data can be called "shared memory". Exemplarily, the memory space is related to one or more of the following information: the maximum video memory space size provided to the heterogeneous computing power card, the active video memory ratio and the adjustment factor; wherein the active video memory ratio is the space occupied by active data in the heterogeneous computing data, and the maximum video memory space ratio. Exemplarily, the space size of the shared memory can be determined according to formula (1): imshared .

[0103]

[0104] Among them, size vmactive Indicates the active video memory ratio mentioned above. In the model training task, the active video memory ratio refers to the video memory space occupied by the active video memory for training hot data. For example, the active video memory ratio is usually around 30%-50%; size vm It represents the maximum video memory space size provided to the heterogeneous computing power card. The physical video memory size of a single heterogeneous computing card, taking NVIDIA's GPU card as an example, is usually 40G or 48G, and 80G, etc.; σ represents the adjustment factor. In the embodiment of the present application, taking into account the fluctuations in the proportion of active video memory and the training time of the business, a design of increasing the adjustment factor is proposed. The value range of this factor is (0,100%), taking a certain percentage. The factor value is configurable. The operation and maintenance personnel of the heterogeneous computing power platform can perform the configuration according to the experience of the training scenario, or make changes and adjustments during the interval of training suspension, and the adjustment can take effect immediately.

[0105] In an exemplary embodiment, the memory space for storing the above-mentioned heterogeneous computing data adopts a shared memory management mode. Among them, shared memory is a method of interprocess communication (IPC) that allows multiple processes to access the same physical memory area. Thus, these processes can directly read and write data in the memory area, thereby realizing efficient data exchange between processes. Exemplarily, shared memory segments are created and mounted at the operating system level, for example, by calling the shmget(), shmat() and other functions of the Portable Operating System Interface of UNIX (POSIX) system call to create and mount shared memory segments.

[0106] For example, memory space is managed using shared memory and pointer addressing. In programming, a pointer is a variable that stores the address of another variable in memory. When memory space is managed using shared memory, each process can access this shared memory area through pointers. For example, after process A creates or connects to a shared memory area, it can assign the starting address of this memory area to a pointer variable and then use this pointer variable to read and write data in the shared memory.

[0107] When using shared memory and pointer addressing to store heterogeneous computing data, concurrency control can be employed. This prevents data contention issues caused by multiple processes simultaneously reading and writing to the same memory location. For example, synchronization mechanisms such as mutexes can be employed. This effectively ensures the smooth execution of heterogeneous computing tasks.

[0108] The embodiment of the present application uses memory to store heterogeneous computing data that has been transferred from the video memory and adopts a shared memory management model to manage it. While effectively improving the video memory space, it can efficiently serve the heterogeneous computing process and ensure the smooth execution of heterogeneous computing tasks.

[0109] In the heterogeneous technology method provided in the embodiments of the present application, the heterogeneous computing process includes at least data exchange between the heterogeneous computing power card and the video memory. To reduce the use of video memory space, heterogeneous computing data stored in the video memory will be transferred to the internal memory when preset conditions are met. Therefore, when the heterogeneous computing power card accesses certain data, the central processing unit determines whether the target object being accessed exists in the video memory based on the video memory address information in the logical mapping table. If the target object does not exist in the video memory, the central processing unit then determines the target object from the internal memory based on the memory address information in the logical mapping table and loads it from the internal memory into the video memory. Furthermore, the central processing unit can provide the target object loaded into the video memory to the heterogeneous computing power card to ensure the smooth execution of the heterogeneous computing process. In the solution provided in the embodiments of the present application, heterogeneous computing data that meets the preset conditions is transferred from the video memory to the internal memory, and a write-to-copy strategy is implemented for the data written to the internal memory. This can expand the storage space for heterogeneous computing data and improve the utilization of the video memory space, thereby achieving video memory overselling and thus facilitating the high-performance computing of the heterogeneous computing power card. The solution provided by this embodiment can also solve the problem of model training being unable to load due to insufficient graphics memory. It can also accelerate the efficiency of model training service access, eliminating the need to reduce the amount of training data per session by adapting graphics memory. This decouples model training design from the service side, contributing to high-quality model training output. It also avoids the need to expand the number of heterogeneous devices to load model training, significantly reducing the cost of heterogeneous computing equipment.

[0110] The above describes in detail the heterogeneous computing method provided by the embodiment of the present application. The following describes the heterogeneous computing method provided by the embodiment of the present application through specific examples.

[0111] Figure 6 This is a flow chart of executing training tasks through heterogeneous computing provided in the embodiment of this application. Figure 6 , the method P600 shown in the figure includes S61 to S616.

[0112] In S61, the training task is produced. In S62, the training task is executed. And, in S63, the training data is loaded. For example, once the heterogeneous computing power container for the training task is produced, the AI ​​model calculation starts running, with training data loading being prioritized during the calculation.

[0113] In S64, the logical mapping table is read. Since the addressing table for the training data is in the logical mapping table, the field information in the mapping table is read and the address information of the video memory and the internal memory is separated. First, a judgment is made based on the video memory address information. If the target object to be accessed has video memory space data, the data is read from the video memory and loaded into the heterogeneous computing process (S65-S67).

[0114] If the target object to be accessed is not in the video memory, it is determined whether it is in the memory. The data in the memory is copied from the memory to the video memory via the bus according to the pointer address of the shared memory, completing the data loading (S68-S610 and S67). For example, after the data is communicated across the bus, to prevent data corruption, a checksum verification operation is performed after the data is copied.

[0115] If the target object to be accessed is not in the video memory or the internal memory, it is further determined whether the target object exists in the process of transferring from the video memory to the internal slave. If the target object exists in the process of transferring from the video memory to the internal slave, the target object is restored to its original storage location in the video memory without updating the logical mapping table (S611-S612 and S67).

[0116] In addition to the above loading process, the data in the video memory will also be polled and judged as cold data. When the preset conditions are met, the data will be cooled from the video memory to the shared memory, and the logical mapping table will be updated to achieve the purpose of video memory overselling (S614-S616).

[0117] The solution provided by the embodiment of the present application transforms the management and control operation surface from directly operating the physical video memory to operating the logical mapping table through a logical mapping table. According to the video memory address information and the memory address information in the logical mapping table, the acquisition operation of the target object to be accessed is implemented. If the target object stored in the video memory meets the preset conditions, it is sunk to the memory and the above-mentioned logical mapping table is updated. Therefore, when the target object in the memory is accessed, the copy-on-write strategy is adopted to achieve overselling and reuse of the video memory, thereby improving the utilization rate of the video memory space. At the same time, the solution provided by this embodiment can solve the problem of model training that cannot be loaded due to insufficient video memory, and can also accelerate the efficiency of model training business access, and there is no need to reduce the amount of single training data by adapting the video memory, decoupling the model training design on the business side, and contributing to the high-quality output of model training. It can also avoid the problem of needing to expand the number of heterogeneous devices to package model training, and intuitively reduce the cost investment of heterogeneous computing devices.

[0118] The embodiment of the present application can realize overselling of physical video memory, referring to Figure 7 In this embodiment of the application, the logical mapping table constructed by the physical video memory can be used as the main body of video memory space allocation. When the computing power container is produced, the space size in the logical mapping table is allocated, for example Figure 7The 40G physical video memory is oversold as 80G video memory in the logical mapping table, and each computing power container is allocated 40G. For example, the original physical video memory can only be provided to one container for heterogeneous computing processes, but through the embodiment of the present application, the original 40G physical video memory can be provided to two containers at the same time to run heterogeneous computing processes in parallel; it can also be understood that the original physical video memory can only be provided to one heterogeneous computing task (such as training model A) for heterogeneous computing processes, but through the embodiment of the present application, the original 40G physical video memory can be provided to two heterogeneous computing tasks (such as training model A and training model B) at the same time to run heterogeneous computing processes in parallel. Therefore, this application also tried to use the copy-on-write technology to reuse 40G of physical video memory space, effectively improving the physical video memory.

[0119] The above is combined with Figures 1 to Figure 7 , describes the method embodiment of the present application in detail, and the following is combined with Figure 8 , describe in detail the device embodiments of the present application.

[0120] Figure 8 This is a schematic diagram of the structure of the heterogeneous computing device 800 provided in an embodiment of the present application. Figure 8 The heterogeneous computing device 800 includes: a first determination module 810, a second determination module 820 and a control module 830; wherein the first determination module 810 is used to determine whether the target object to be accessed by the heterogeneous computing power card exists in the video memory, wherein the heterogeneous computing process at least includes data exchange between the heterogeneous computing power card and the video memory, and the heterogeneous computing data stored in the video memory will be transferred to the memory when the preset conditions are met; the second determination module 820 is used to determine the target object from the memory when the target object does not exist in the video memory, and load the target object from the memory into the video memory; and the control module 830 is used to provide the target object in the video memory to the heterogeneous computing power card.

[0121] In an exemplary embodiment, based on the above solution, the above preset conditions include any one of the following: no access within a first preset time period and the access frequency within a second preset time period is less than a preset threshold.

[0122] In an exemplary embodiment, based on the above scheme, the above control module 830 is also used to: when it is determined that the heterogeneous computing data stored in the above video memory meets the above preset conditions, it will be transferred to the internal memory; and after the heterogeneous computing data stored in the above video memory is transferred to the internal memory, the storage space of the transferred data in the above video memory will be set as visible space.

[0123] In an exemplary embodiment, based on the above scheme, the video memory address information and memory address information of the above heterogeneous computing data are stored through a logical mapping table; wherein the above video memory address information is the current storage address of the above heterogeneous computing data in the above video memory, and the above memory address information is the current storage address of the above heterogeneous computing data in the above memory.

[0124] In an exemplary embodiment, based on the above scheme, the above-mentioned heterogeneous computing device 800 also includes: an update module; wherein the above-mentioned update module is used to: update the video memory address information and the memory address information in the above-mentioned logical mapping table when the heterogeneous computing data is transferred from the above-mentioned video memory to the above-mentioned memory, or when the heterogeneous computing data stored in the above-mentioned memory is accessed.

[0125] In an exemplary embodiment, based on the above scheme, the above-mentioned update module is specifically used to: if the above-mentioned heterogeneous computing data is stored in the above-mentioned video memory and meets the above-mentioned preset conditions, then the above-mentioned heterogeneous computing data is transferred to the internal memory; if the above-mentioned heterogeneous computing data is not accessed during the transfer process, then the video memory address information and the internal memory address information after the transfer are updated to the above-mentioned logical mapping table; the above-mentioned update module is also used to: if the above-mentioned heterogeneous computing data is accessed during the transfer process, then the above-mentioned heterogeneous computing data is kept unchanged in the above-mentioned video memory, and the video memory address information and the internal memory address information before the transfer in the logical mapping table are kept unchanged; the above-mentioned heterogeneous computing device 800 also includes: a timing module; wherein the above-mentioned timing module is used to, when the heterogeneous computing data is accessed during the transfer process, re-time the length of time that the above-mentioned heterogeneous computing data is not accessed.

[0126] In an exemplary embodiment, based on the above scheme, if the above heterogeneous computing data is stored in the above memory and is accessed, the above control module is also specifically used to transfer the above heterogeneous computing data from the above memory to the above video memory, and the above update module is also specifically used to update the transferred video memory address information and memory address information to the above logical mapping table.

[0127] In an exemplary embodiment, based on the above solution, the check code of the heterogeneous computing data transferred from the video memory to the internal memory is also saved through the above logical mapping table;

[0128] The control module specifically includes: transferring the heterogeneous computing data from the memory to the video memory via the bus;

[0129] The above-mentioned heterogeneous computing device 800 also includes: a verification module; wherein the above-mentioned verification module is used to: verify the mechanical computing data transferred to the above-mentioned video memory based on the verification code of the above-mentioned transferred heterogeneous computing data; wherein, if the verification passes, the above-mentioned heterogeneous computing data in the above-mentioned video memory is provided to the above-mentioned heterogeneous computing power card.

[0130] In an exemplary embodiment, based on the above scheme, the above-mentioned first determination module 810 is specifically used to: determine the storage location of the above-mentioned heterogeneous computing data in the above-mentioned video memory according to the video memory address information stored in the above-mentioned logical mapping table; and, based on the above-mentioned storage location of the above-mentioned heterogeneous computing data in the above-mentioned video memory, determine whether the target object to be accessed by the above-mentioned heterogeneous computing power card exists in the above-mentioned video memory.

[0131] In an exemplary embodiment, based on the above scheme, the above-mentioned second determination module 820 is specifically used to: determine the storage location of the heterogeneous computing data in the above-mentioned memory according to the memory address information stored in the above-mentioned logical mapping table; and, based on the above-mentioned storage location of the above-mentioned heterogeneous computing data in the above-mentioned memory, determine the above-mentioned target object from the above-mentioned memory; wherein the above-mentioned memory address information includes at least one pair of address pointer values, and each pair of the above-mentioned address pointer values ​​includes a memory address start pointer value and a memory address end pointer value.

[0132] In an exemplary embodiment, based on the above scheme, the above memory adopts a shared memory management mode; the space of the above memory is related to one or more of the following information: the maximum video memory space size provided to the above heterogeneous computing power card, the active video memory ratio and the adjustment factor; wherein, the active video memory ratio is the space occupied by active data in the above heterogeneous computing data, and the above maximum video memory space ratio.

[0133] It should be understood that the heterogeneous computing device embodiment and the heterogeneous computing method embodiment may correspond to each other, and similar descriptions can refer to the method embodiment. To avoid repetition, they will not be described here. Specifically, Figure 8 The heterogeneous computing device shown can execute the embodiments of the above-mentioned heterogeneous computing method, and the aforementioned and other operations and / or functions of each module in the device are respectively for implementing the embodiments of the heterogeneous computing method, which will not be repeated here for the sake of brevity.

[0134] In the solution provided by the embodiments of the present application, the heterogeneous computing process includes at least data exchange between the heterogeneous computing power card and the video memory. To reduce the use of video memory space, heterogeneous computing data stored in the video memory will be transferred to the internal memory when preset conditions are met. Therefore, when the heterogeneous computing power card accesses certain data, the central processing unit determines whether the target object being accessed exists in the video memory based on the video memory address information in the logical mapping table. If the target object does not exist in the video memory, the central processing unit then determines the target object from the internal memory based on the memory address information in the logical mapping table and loads it from the internal memory into the video memory. Furthermore, the central processing unit can provide the target object loaded into the video memory to the heterogeneous computing power card to ensure the smooth execution of the heterogeneous computing process. In the solution provided by the embodiments of the present application, heterogeneous computing data that meets the preset conditions is transferred from the video memory to the internal memory, and a write-to-copy strategy is implemented for the data written to the internal memory. This can expand the storage space for heterogeneous computing data and improve the utilization of the video memory space, thereby achieving video memory overselling and thus facilitating the high-performance computing of the heterogeneous computing power card. The solution provided by this embodiment can also solve the problem of model training being unable to load due to insufficient graphics memory. It can also accelerate the efficiency of model training service access, eliminating the need to reduce the amount of training data per session by adapting graphics memory. This decouples model training design from the service side, contributing to high-quality model training output. It also avoids the need to expand the number of heterogeneous devices to load model training, significantly reducing the cost of heterogeneous computing equipment.

[0135] The above describes the apparatus of the heterogeneous computing method embodiment of the present application from the perspective of functional modules in conjunction with the accompanying drawings. It should be understood that the functional module can be implemented in hardware form, can be implemented by instructions in software form, or can be implemented by a combination of hardware and software modules. Specifically, the steps of the method embodiment in the embodiment of the present application can be completed by the hardware integrated logic circuit and / or software instructions in the processor. The steps of the method disclosed in the embodiment of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. Optionally, the software module can be located in a mature storage medium in the art such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps in the above method embodiment in combination with its hardware.

[0136] Figure 9 is a schematic block diagram of a heterogeneous computing device 900 provided in an embodiment of the present application. Figure 9 The heterogeneous computing device 900 can be used to perform the above-mentioned heterogeneous computing method. Figure 9 As shown, the heterogeneous computing device 900 may include:

[0137] The memory 910 and the processor 920 are configured to store a computer program 930 and transmit the program code 930 to the processor 920. In other words, the processor 920 can call and execute the computer program 930 from the memory 910 to implement the method in the embodiment of the present application.

[0138] For example, the processor 920 may be configured to execute the steps in the above method according to the instructions in the computer program 930 .

[0139] In some embodiments of the present application, the memory 910 includes a video memory and a memory. The video memory is used to store heterogeneous computing data; the memory is used to store heterogeneous computing data transferred from the video memory. The processor 920 includes a central processing unit and a heterogeneous computing card, wherein the heterogeneous computing power card is used to exchange data with the video memory to perform heterogeneous computing; the central processing unit is used to receive an access request from the heterogeneous computing power card, and when it is determined that the target object to be accessed does not exist in the video memory, it determines the target object from the memory and loads the target object from the memory into the video memory, and provides the target object in the video memory to the heterogeneous computing power card. The above-mentioned heterogeneous computing power card is a hardware acceleration card or component for implementing a heterogeneous computing process, which may be a neural network processing unit (NPU), a graph processing unit (GPU), a field programmable gate array (FPGA), a digital signal processing (DSP) chip or other accelerator optimized for specific tasks.

[0140] In some embodiments of the present application, the processor 920 may include but is not limited to:

[0141] General-purpose processor, Digital Signal Processor (DSP), Application Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc.

[0142] In some embodiments of the present application, the memory 910 includes but is not limited to:

[0143] Volatile memory and / or non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0144] In some embodiments of the present application, the computer program 930 may be divided into one or more modules, which are stored in the memory 910 and executed by the processor 920 to complete the heterogeneous computing method provided herein, or to perform the heterogeneous computing method. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program 930 in the heterogeneous computing device.

[0145] like Figure 9 As shown, the heterogeneous computing device 900 may further include:

[0146] The transceiver 940 may be connected to the processor 920 or the memory 910 .

[0147] The processor 920 may control the transceiver 940 to communicate with other devices. Specifically, the processor 920 may send information or data to other devices or receive information or data sent by other devices. The transceiver 940 may include a transmitter and a receiver. The transceiver 940 may further include an antenna, which may be one or more.

[0148] It should be understood that the various components in the heterogeneous computing device 930 are connected via a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus, and a status signal bus.

[0149] According to one aspect of the present application, a computer storage medium is provided, on which a computer program is stored. When the computer program is executed by a computer, the computer is enabled to perform the method of the above-described method embodiment. Alternatively, the present application also provides a computer program product containing instructions. When the computer is executed, the instructions cause the computer to perform the heterogeneous computing method of the above-described method embodiment.

[0150] According to another aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the heterogeneous computing method of the above-described method embodiment.

[0151] In other words, when implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (DVD)), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0152] Those skilled in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0153] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules is merely a logical function division. In actual implementation, there may be other division methods, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0154] Modules described as separate components may or may not be physically separate, and components displayed as modules may or may not be physical modules, i.e., they may be located in one place or distributed across multiple network elements. Some or all of the modules may be selected based on actual needs to achieve the purpose of the present embodiment. For example, the functional modules in the various embodiments of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into a single module.

[0155] The above content is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A heterogeneous computing method, characterized in that: The method comprises: Determine whether the target object to be accessed by the heterogeneous computing power card exists in the video memory, wherein the heterogeneous computing data stored in the video memory will be transferred to the internal memory if a preset condition is met; In the case that the target object does not exist in the video memory, determining the target object from the memory, and loading the target object from the memory into the video memory; Providing the target object in the video memory to the heterogeneous computing power card.

2. The method according to claim 1, characterized in that The preset condition includes any one of the following: no access within a first preset time period and the access frequency within a second preset time period is less than a preset threshold.

3. The method according to claim 1, characterized in that in, The method further comprises: When it is determined that the heterogeneous computing data stored in the video memory meets the preset conditions, it will be transferred to the internal memory, and after the heterogeneous computing data stored in the video memory is transferred to the internal memory, the storage space of the transferred data in the video memory is set as visible space.

4. The method according to claim 1, wherein Storing the video memory address information and the internal memory address information of the heterogeneous computing data through a logical mapping table; The video memory address information is the current storage address of the heterogeneous computing data in the video memory, and the memory address information is the current storage address of the heterogeneous computing data in the memory.

5. The method according to claim 4, characterized in that The method further comprises: When heterogeneous computing data is transferred from the video memory to the internal memory, or when heterogeneous computing data stored in the internal memory is accessed, the video memory address information and the internal memory address information in the logical mapping table are updated.

6. The method according to claim 5, characterized in that The updating of the video memory address information and the internal memory address information in the logical mapping table includes: If the heterogeneous computing data is stored in the video memory and meets the preset condition, the heterogeneous computing data is transferred to the memory; If the heterogeneous computing data is not accessed during the transfer process, updating the transferred video memory address information and internal memory address information to the logical mapping table; If the heterogeneous computing data is accessed during the transfer process, the heterogeneous computing data is kept unchanged in the video memory, and the video memory address information and the memory address information before the transfer in the logical mapping table are kept unchanged, and the time period during which the heterogeneous computing data is not accessed is retimed.

7. The method according to claim 5, characterized in that The updating of the video memory address information and the internal memory address information in the logical mapping table includes: If the heterogeneous computing data is stored in the memory and is accessed, the heterogeneous computing data is transferred from the memory to the video memory, and the transferred video memory address information and memory address information are updated to the logical mapping table.

8. The method according to claim 7, characterized in that The logic mapping table also stores the check code of the heterogeneous computing data transferred from the video memory to the internal memory; The transferring the heterogeneous computing data from the memory to the video memory includes: Transferring the heterogeneous computing data from the memory to the video memory via a bus; The method further comprises: Verifying the mechanism computing data transferred to the video memory according to the verification code of the transferred heterogeneous computing data; Wherein, if the verification passes, the heterogeneous computing data in the video memory is provided to the heterogeneous computing power card.

9. The method according to any one of claims 4 to 8, characterized in that Determining whether the target object to be accessed by the heterogeneous computing power card exists in the video memory includes: Determining a storage location of the heterogeneous computing data in the video memory according to the video memory address information stored in the logic mapping table; Based on the storage location of the heterogeneous computing data in the video memory, it is determined whether the target object to be accessed by the heterogeneous computing power card exists in the video memory.

10. The method according to any one of claims 4 to 8, characterized in that The determining the target object from the memory includes: Determining a storage location of the heterogeneous computing data in the memory according to the memory address information stored in the logical mapping table; Determining the target object from the memory based on the storage location of the heterogeneous computing data in the memory; The memory address information includes at least one pair of address pointer values, and each pair of address pointer values ​​includes a memory address start pointer value and a memory address end pointer value.

11. The method according to any one of claims 1 to 8, characterized in that The memory adopts a shared memory management mode; the memory space is related to one or more of the following information: the maximum video memory space size provided to the heterogeneous computing power card, the active video memory ratio and the adjustment factor; The active video memory ratio is the ratio of the space occupied by active data in the heterogeneous computing data to the maximum video memory space.

12. A heterogeneous computing device, characterized in that: The device comprises: A first determination module is configured to determine whether a target object to be accessed by a heterogeneous computing power card exists in a video memory, wherein the heterogeneous computing process at least includes data exchange between the heterogeneous computing power card and the video memory, and the heterogeneous computing data stored in the video memory will be transferred to the internal memory if a preset condition is met; A second determining module is configured to determine the target object from the memory and load the target object from the memory into the video memory if the target object does not exist in the video memory; A control module is configured to provide the target object in the video memory to the heterogeneous computing power card.

13. A heterogeneous computing device, characterized in that: The device comprises: Video memory, used to store heterogeneous computing data; A memory for storing heterogeneous computing data transferred from the video memory; A heterogeneous computing power card, used to exchange data with the video memory to perform heterogeneous computing; A central processing unit is configured to receive an access request from the heterogeneous computing power card, determine the target object from the memory when it is determined that the target object to be accessed does not exist in the video memory, load the target object from the memory into the video memory, and provide the target object in the video memory to the heterogeneous computing power card.

14. A heterogeneous computing device comprising a processor and a memory; The memory is used to store computer programs; The processor is configured to execute the computer program to implement the heterogeneous computing method as described in any one of claims 1 to 11.

15. A computer-readable storage medium, characterized in that For storing computer programs; The computer program enables a computer to execute the heterogeneous computing method according to any one of claims 1 to 11.

Images