Heterogeneous resource processing method and device among multiple systems, computer equipment, readable storage medium and program product

Abstract

The invention relates to a heterogeneous resource processing method and device among multiple systems, computer equipment, a readable storage medium and a program product. A target virtual address of a target heterogeneous resource in a second system sent by a first system is obtained through heterogeneous equipment storing mirror image heterogeneous resources of the first system and the second system, and after a target mirror image heterogeneous resource is determined based on the target virtual address, the target mirror image heterogeneous resource is synchronized to a mirror image storage space of the first system. And enabling the first system to process the target mirror image heterogeneous resource in the mirror image storage space to obtain a corresponding processing result. Compared with a traditional method that heterogeneous resources are transmitted through message packaging and a kernel protocol stack, the method has the advantages that the heterogeneous devices connected with all the systems are constructed, the mirror image heterogeneous resources of all the systems are managed through the heterogeneous devices, and all the systems can transmit the mirror image heterogeneous resources through the heterogeneous devices; and calling and synchronization of mirror image heterogeneous resources among different systems are carried out in combination with the virtual addresses, so that the technical effect of improving the efficiency of heterogeneous resource processing among multiple systems is realized.

Description

Technical Field

[0001] The present application relates to the technical field of data processing, and in particular, to a method, device, computer device, computer-readable storage medium, and computer program product for processing heterogeneous resources between multiple systems. Background Art

[0002] With the development of data processing technology in the computer field, it has become mainstream to use multiple systems for resource processing. The resources between each system are heterogeneous resources and have physical isolation. Currently, when a system needs to use heterogeneous resources of another system, it usually encapsulates and parses special messages and transmits resources in the form of network messages through the kernel protocol stack. However, the transmission efficiency of heterogeneous resources through message encapsulation and the kernel protocol stack is low, thus reducing the efficiency of processing heterogeneous resources.

[0003] Therefore, there is a defect of low processing efficiency in the current processing of heterogeneous resources between multiple systems. Summary of the Invention

[0004] Based on this, it is necessary to provide a method, device, computer device, computer-readable storage medium, and computer program product for processing heterogeneous resources between multiple systems that can improve processing efficiency in view of the above technical problems.

[0005] In a first aspect, the present application provides a method for processing heterogeneous resources between multiple systems, which is applied to a heterogeneous device and includes:

[0006] Obtain a target virtual address corresponding to a target heterogeneous resource in a second system sent by a first system; the heterogeneous device is connected to the first system and the second system respectively through a heterogeneous resource interface; the heterogeneous device stores one or more mirror heterogeneous resources; each mirror heterogeneous resource belongs to the first system or the second system;

[0007] Determine a target mirror heterogeneous resource corresponding to the target heterogeneous resource according to the target virtual address;

[0008] Synchronize the target mirror heterogeneous resource to a mirror storage space in the first system; the first system is used to process the target mirror heterogeneous resource in the mirror storage space to obtain a corresponding processing result.

[0009] In a second aspect, the present application further provides a device for processing heterogeneous resources between multiple systems, which is applied to a heterogeneous device and includes:

[0010] An acquisition module, configured to acquire a target virtual address corresponding to a target heterogeneous resource in a second system sent by a first system; the heterogeneous device is respectively connected to the first system and the second system through a heterogeneous resource interface; the heterogeneous device stores one or more mirrored heterogeneous resources; each of the mirrored heterogeneous resources belongs to the first system or the second system;

[0011] A determination module, configured to determine a target mirrored heterogeneous resource corresponding to the target heterogeneous resource according to the target virtual address;

[0012] A processing module, configured to synchronize the target mirrored heterogeneous resource to a mirrored storage space in the first system; the first system is configured to process the target mirrored heterogeneous resource in the mirrored storage space to obtain a corresponding processing result.

[0013] In a third aspect, the present application further provides a computer device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the steps of the above method are implemented.

[0014] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0015] In a fifth aspect, the present application further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0016] For the above heterogeneous resource processing method, device, computer device, computer-readable storage medium, and computer program product among multiple systems, through a heterogeneous device storing mirrored heterogeneous resources of a first system and a second system, the target virtual address of the target heterogeneous resource in the second system sent by the first system is acquired. After determining the target mirrored heterogeneous resource based on the target virtual address, it is synchronized to the mirrored storage space of the first system, enabling the first system to process the target mirrored heterogeneous resource in the mirrored storage space to obtain a corresponding processing result. Compared with the traditional transmission of heterogeneous resources through message encapsulation and the kernel protocol stack, this solution constructs a heterogeneous device connected to each system, manages the mirrored heterogeneous resources of each system through the heterogeneous device, and each system can call and synchronize the mirrored heterogeneous resources between different systems through the heterogeneous device in combination with the virtual address, achieving the technical effect of improving the efficiency of heterogeneous resource processing among multiple systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] To more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the accompanying drawings required for the description of the embodiments of the present application or related technologies. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0018] Figure 1 It is an application environment diagram of a heterogeneous resource processing method between multiple systems in an embodiment;

[0019] Figure 2 It is a schematic flowchart of a heterogeneous resource processing method between multiple systems in an embodiment;

[0020] Figure 3 It is a schematic flowchart of a heterogeneous resource processing method between multiple systems in another embodiment;

[0021] Figure 4 It is a structural block diagram of a heterogeneous resource processing device between multiple systems in an embodiment;

[0022] Figure 5 It is an internal structure diagram of a computer device in an embodiment. Detailed implementation manners

[0023] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0024] In related technologies, due to the physical isolation of different devices, it is usually impossible to directly share heterogeneous resources such as memory between multiple Linux systems. However, there may be certain data dependencies during operation. Data is usually transmitted between multiple systems in the form of network packets, such as common network transmission protocols such as TCP (Transmission Control Protocol) and UDP (User Datagram Protocol).

[0025] For example, for two Linux systems A and B, when system A needs to obtain data b on system B, the usual implementation solution is as follows: First, the message transmission format between systems A and B is agreed upon. Then, through configuration or other means, the IP address ip_b of system B, the listening port port_b, and the transmission protocol tcp / udp used are informed to system A. Then system B also needs to listen for corresponding protocol messages on the agreed IP address ip_b and port_b. When system A needs to obtain data b on system B, it needs to first fill in the corresponding message fields according to the agreed message format, such as information about this being a read operation, the position to be read, the length, etc. Then, the message is sent to the specified port port_b of the specified IP address ip_b using the agreed transmission protocol via the kernel network protocol stack. When system B receives the above message, it parses out that this is a read operation and information such as the position and length to be read according to the agreed message format. And it finds the data to be read on this system, then encapsulates it into a message in the agreed format, and finally sends it to system A through the same path. When system A receives the above message, it parses out the data b to be read according to the agreed message format and saves it locally for subsequent use.

[0026] For the above data interaction and processing method between systems, on the one hand, this data transmission method needs to pass through the kernel protocol stack, and the data transmission efficiency is not high. On the other hand, the data to be transmitted needs to go through special message encapsulation and parsing, which is not convenient to use.

[0027] Based on this, the present application constructs heterogeneous devices connected to each system, manages the mirror heterogeneous resources of each system through the heterogeneous devices, and each system can, through the heterogeneous devices, combine virtual addresses to call and synchronize the mirror heterogeneous resources between different systems, achieving the technical effect of improving the efficiency of heterogeneous resource processing among multiple systems.

[0028] The heterogeneous resource processing method between multiple systems provided by the embodiments of the present application can be applied to, for example Figure 1In the application environment shown. Among them, heterogeneous devices communicate with each system through a network or an FPGA (Field Programmable Gate Array). Among them, each system includes modules such as a user-state API (Application Programming Interface), heterogeneous resource mapping, heterogeneous owner migration, heterogeneous data synchronization, heterogeneous address space management, and a heterogeneous resource abstraction layer. The heterogeneous device can manage the mirror heterogeneous resources of each heterogeneous resource in each system. For example, the first system can call the target mirror heterogeneous resource in the second system through the heterogeneous device. Among them, the heterogeneous device, the first system, and the second system can all be independent physical servers, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0029] In an exemplary embodiment, as Figure 2 shown, a method for processing heterogeneous resources between multiple systems is provided. Taking the heterogeneous device in Figure 1 as an example, the following steps S202 to S206 are included. Among them:

[0030] Step S202, obtain the target virtual address corresponding to the target heterogeneous resource in the second system sent by the first system; the above-mentioned heterogeneous device is respectively connected to the above-mentioned first system and the above-mentioned second system through a heterogeneous resource interface; the above-mentioned heterogeneous device stores one or more mirror heterogeneous resources; each of the above-mentioned mirror heterogeneous resources belongs to the above-mentioned first system or the above-mentioned second system.

[0031] Among them, the first system and the second system can be different systems. For example, they can be Linux systems, and the second system can include multiple ones. During the data processing process of the first system and the second system, they can use the heterogeneous resources of other systems, and the first system and the second system itself can also contain heterogeneous resources that other systems need to use. The above-mentioned heterogeneous resources can be read-write heterogeneous resources. To achieve the sharing of heterogeneous resources between multiple systems, the above-mentioned first system and the second system can manage the resources.

[0032] As Figure 1 shown, the heterogeneous device can be a device connected to the first system and the second system, used to realize the sharing of heterogeneous resources between the first system and the second system. And the first system and the second system can build corresponding modules as the basis for sharing heterogeneous resources. Among them, each system in the above-mentioned first system and the second system can include modules such as a heterogeneous resource abstraction layer, heterogeneous address space management, heterogeneous resource mapping, heterogeneous owner migration, heterogeneous data synchronization, and a user-state API.

[0033] Among them, heterogeneous resources refer to device resources of other systems that exist in multiple systems and cannot be directly accessed by one system, such as memory information on other systems. The heterogeneous resource abstraction layer refers to a software adaptation layer that shields specific hardware differences. This adaptation layer provides a unified operation interface for heterogeneous resources to the upper layer, while shielding the implementation differences of different underlying hardware. Among them, the unified operation interface for heterogeneous resources includes, but is not limited to, read and write interfaces for heterogeneous resources. The above-mentioned different hardware refers to the physical channels used to access heterogeneous resources, such as customized FPGA links, directly connected network card devices, etc.

[0034] Among them, the address space refers to the global addressing address of heterogeneous resources, the mirrored memory of heterogeneous resources in the system, the kernel-mode virtual address, the user-mode virtual address, and the kernel and process spaces to which they belong. The mirrored memory of heterogeneous resources in the system refers to the physical memory allocated in the system, which is used to store the data mirror of real heterogeneous resources and can be directly read and written by upper-layer users. The size of the physical memory allocated by the system can be a subset of the real heterogeneous resource space, with physical pages as the unit and does not require physical pages to be continuous. Heterogeneous owner migration means migrating the owner of heterogeneous resources. Heterogeneous resource data synchronization means synchronizing heterogeneous resources among various systems. User-mode API refers to the function interfaces such as mapping, migration, and synchronization of heterogeneous resources provided in the user-mode space.

[0035] The above-mentioned heterogeneous devices can be connected to the first system and the second system respectively through the heterogeneous resource interface, and one or more mirrored heterogeneous resources can be stored in the above-mentioned heterogeneous devices, and each mirrored heterogeneous resource belongs to the first system or the second system. Among them, the above-mentioned heterogeneous resource interface can be connected based on one or more of FPGA and network cards, and the mirrored heterogeneous resources in the above-mentioned heterogeneous devices can be the mirrored data of heterogeneous resources in each system. When the first system needs to call the heterogeneous resources in the second system, it can send the corresponding target virtual address to the heterogeneous device. Among them, the target virtual address represents the virtual address of the target heterogeneous resource that the first system needs to call, and the heterogeneous device can pre-arrange and manage the addresses of each heterogeneous resource in advance, so that each heterogeneous resource has a corresponding virtual address for calling. Thus, the heterogeneous device can obtain the target virtual address corresponding to the target heterogeneous resource in the second system sent by the first system.

[0036] Step S204, determine the target mirrored heterogeneous resource corresponding to the above-mentioned target heterogeneous resource according to the above-mentioned target virtual address.

[0037] Among them, the heterogeneous device pre-arranges the addresses of heterogeneous resources in each system in advance, and each mirrored heterogeneous resource corresponding to each heterogeneous resource in the heterogeneous device has a corresponding virtual address. That is, there is a mapping relationship between the mirrored heterogeneous resource and the virtual address. The heterogeneous device can determine the target mirrored heterogeneous resource corresponding to the target heterogeneous resource according to the above-mentioned target virtual address.

[0038] Step S206: Synchronize the above-mentioned target mirror heterogeneous resources to the mirror storage space in the above-mentioned first system; the above-mentioned first system is used to process the above-mentioned target mirror heterogeneous resources in the mirror storage space to obtain corresponding processing results.

[0039] Among them, the heterogeneous device can synchronize the target mirror heterogeneous resources to the mirror storage space in the above-mentioned first system based on the request of the first system for the target mirror heterogeneous resources. Among them, the above-mentioned mirror storage space can be the space for storing mirror resources in each system, for example, it can be the local mirror memory in the system.

[0040] After receiving the target mirror heterogeneous resources, the first system can process the target mirror heterogeneous resources in the mirror storage space to obtain corresponding processing results. For example, the user mode or kernel mode in the first system can perform corresponding calls and processing on the target mirror heterogeneous resources. Among them, the above-mentioned processing can be processing such as modifying the target mirror heterogeneous resources, and thus the processing results can include the processed mirror heterogeneous resources.

[0041] Among them, the above-mentioned kernel mode represents the level where the system itself runs; each system can also include a user mode API. Among them, the user mode represents the level where each application in the system is located, and the user mode API refers to function interfaces such as heterogeneous resource mapping, migration, and synchronization provided in the user mode space. All associated systems can directly map, read and write, and manage heterogeneous resource data through the above-mentioned user mode API, thereby realizing shared access to heterogeneous resources among multiple systems.

[0042] Moreover, since all systems have a unified heterogeneous address space addressing, it is thus allowed to directly share data structures containing pointers among multiple systems, just like accessing the same memory space within the same process. For example, when system A needs to obtain data b on system B. At this time, only a data synchronization operation needs to be performed, and after this operation, the mirror data on system A will be consistent with system B, and the data structure defined by system B on this heterogeneous memory can be directly read and written. During this period, it will not pass through the network protocol stack, and there is no need to perform message encapsulation and parsing anymore. Thereby improving the collaborative access efficiency and usability of read and write type heterogeneous resources among multiple systems.

[0043] In the above heterogeneous resource processing method between multiple systems, through a heterogeneous device storing the mirror heterogeneous resources of the first system and the second system, the target virtual address of the target heterogeneous resource in the second system sent by the first system is obtained. After determining the target mirror heterogeneous resource based on the target virtual address, it is synchronized to the mirror storage space of the first system, so that the first system processes the target mirror heterogeneous resource in the mirror storage space to obtain the corresponding processing result. Compared with the traditional transmission of heterogeneous resources through message encapsulation and the kernel protocol stack, this solution constructs a heterogeneous device connected to each system, manages the mirror heterogeneous resources of each system through the heterogeneous device, and each system can use the heterogeneous device to combine the virtual address to call and synchronize the mirror heterogeneous resources between different systems, achieving the technical effect of improving the efficiency of heterogeneous resource processing between multiple systems.

[0044] In one embodiment, before obtaining the target virtual address corresponding to the target heterogeneous resource in the second system sent by the first system, it further includes: obtaining the respective virtual addresses corresponding to the respective heterogeneous resources sent by the above first system and / or the above second system; each of the above virtual addresses is within a preset virtual address range; mapping the respective mirror heterogeneous resources corresponding to the respective heterogeneous resources to the respective above virtual addresses to obtain the mapping relationship between the respective above virtual addresses and the respective above mirror heterogeneous resources.

[0045] In this embodiment, the heterogeneous device can pre-map the virtual addresses of the respective heterogeneous resources in the first system and the second system. Among them, the above virtual address can be an address within a preset virtual address range. Each of the above systems can pre-allocate virtual addresses for the respective heterogeneous resources in its own system. For example, based on the preset virtual address range and in combination with the resource type of the heterogeneous resource, the corresponding virtual address is allocated to each heterogeneous resource. Each system can send the respective virtual addresses corresponding to the respective heterogeneous resources to the heterogeneous device, so that the heterogeneous device can map the respective mirror heterogeneous resources corresponding to the respective heterogeneous resources to the respective corresponding virtual addresses, thereby establishing the mapping relationship between each virtual address and each mirror heterogeneous resource.

[0046] Specifically, the heterogeneous device can perform address space management. For example, the heterogeneous device first globally addresses the heterogeneous resources. Secondly, the same preset virtual address range of the kernel state and the user state is allocated to the heterogeneous resources in different systems. Then, the heterogeneous resource mirror memory is mapped to the kernel state virtual address or the user state virtual address as needed and its mapping relationship is maintained. Thus, the kernel state and the user state in the system can directly call the corresponding heterogeneous resources through the virtual address.

[0047] Through this embodiment, the heterogeneous device can allocate addresses for each heterogeneous resource within a preset virtual address range, so that each system can call the corresponding heterogeneous resource based on the virtual address, improving the processing efficiency of the heterogeneous resource.

[0048] In one embodiment, determining the target mirror heterogeneous resource corresponding to the target heterogeneous resource according to the target virtual address includes: determining a starting call address and an ending call address according to the target virtual address; determining a called interval from the mirror heterogeneous resources corresponding to the target heterogeneous resource as the target mirror heterogeneous resource according to the starting call address and the ending call address.

[0049] In this embodiment, each system can call heterogeneous resources based on the virtual address. When calling heterogeneous resources, the system can select a certain interval of heterogeneous resources to call or call the complete heterogeneous resource.

[0050] The heterogeneous device can determine a starting call address and an ending call address according to the target virtual address. The starting call address represents the virtual address corresponding to the call starting point of the mirror heterogeneous resource, and the ending call address represents the virtual address corresponding to the call ending point of the mirror heterogeneous resource. There is a data interval between the starting call address and the ending call address. Thus, the heterogeneous device can determine a called interval from the mirror heterogeneous resources corresponding to the target heterogeneous resource as the target mirror heterogeneous resource according to the starting call address and the ending call address. That is, the system can achieve partial calls to heterogeneous resources.

[0051] Specifically, the above call process includes synchronizing the target mirror heterogeneous resource in the heterogeneous device to the corresponding system. For example, synchronizing the mirror heterogeneous resource to the local mirror memory. When calling the target mirror heterogeneous resource, the system first obtains the address information of the heterogeneous resource to be synchronized, that is, the target virtual address, from the heterogeneous address space, and then calls the read interface provided by the heterogeneous resource abstraction layer to read the target mirror heterogeneous resource from the heterogeneous device to the local mirror memory. During the process of writing the target mirror heterogeneous resource to the local mirror memory, concurrent protection is supported to prevent other threads in the system from reading incomplete data.

[0052] Heterogeneous resource data synchronization can also be performed between systems. Data synchronization refers to synchronizing the heterogeneous resource data of other systems to the memory space of this system, or synchronizing the memory data of this system to the heterogeneous device. There is no permission control for the former, and only the system with the owner permission is allowed to execute the latter. The data synchronization operation allows specifying the starting address of the resource to be synchronized. For example, the heterogeneous device determines the starting call address and the ending call address of the heterogeneous resource to be synchronized based on the target virtual address, determines the specified data interval as the target mirror heterogeneous resource, that is, only synchronizing the specified data interval is supported.

[0053] The above data synchronization operation supports concurrency control functions, including read-read concurrency and read-write mutual exclusion for synchronization operations with other systems, as well as read-read concurrency and read-write mutual exclusion between the data synchronization process and local threads. This avoids the problem of data being read by other processes when it is being modified halfway, thereby ensuring that each participant sees complete data. Additionally, the above data synchronization operation also supports truly synchronizing data only when the data is updated, in order to reduce unnecessary data synchronization consumption.

[0054] Through this embodiment, heterogeneous devices can determine the corresponding data range to be called based on the target virtual address. When only a specified data range needs to be processed, it is not necessary to call the complete mirror heterogeneous resources, thereby improving the processing efficiency when processing the specified mirror heterogeneous resources.

[0055] In one embodiment, after synchronizing the above target mirror heterogeneous resources to the mirror storage space in the above first system, it further includes: obtaining the processed target mirror heterogeneous resources sent by the above first system according to the above processing result; storing the processed above target mirror heterogeneous resources; the processed above target mirror heterogeneous resources are used to instruct the above second system to synchronize the processed above target mirror heterogeneous resources.

[0056] The above first system that calls heterogeneous resources can process the called target mirror heterogeneous resources, for example, it can be read-write and other processing, and then obtain the processed target mirror heterogeneous resources. The first system can synchronize the processed target mirror heterogeneous resources among various systems, then the first system can send the processed target mirror heterogeneous resources to the heterogeneous device. After the heterogeneous device obtains the processed target mirror heterogeneous resources sent by the first system according to the above processing result, it stores the processed target mirror heterogeneous resources, for example, stores them in the heterogeneous device. Among them, the processed above target mirror heterogeneous resources are used to instruct the above second system to synchronize the processed above target mirror heterogeneous resources. For example, when the second system detects that there are processed target mirror heterogeneous resources synchronized by the first system in the heterogeneous device, it can synchronize the processed target mirror heterogeneous resources to the corresponding mirror storage space in the second system, for example, store them in the mirror memory space, to achieve the synchronization of heterogeneous resources among various systems.

[0057] Specifically, taking the example of the first system invoking the target mirror heterogeneous resources of the second system, the first system can perform customized read and write operations on the invoked target mirror heterogeneous resources according to the business processing logic, and then obtain the processed target mirror heterogeneous resources. The target mirror heterogeneous resources read and written by the first system can be a piece of mirror memory. After the modification is completed, the processed target mirror heterogeneous resources can be submitted to the heterogeneous device. For example, the first system first obtains the address information of the heterogeneous resources to be submitted from the heterogeneous address space, that is, the virtual address of the processed target mirror heterogeneous resources, and then invokes the write interface provided by the heterogeneous resource abstraction layer to write the data in the local mirror memory to the heterogeneous device. Among them, during the process of writing the heterogeneous data to the heterogeneous device, the heterogeneous device can support enabling concurrent protection to prevent other systems from reading incomplete data.

[0058] In addition, if the system submitting the processed target mirror heterogeneous resources does not have the owner permission, the submission operation will return a failure. Here, the owner refers to the owner of the heterogeneous resources. Taking the operating system as the granularity, a non-owner system can read the heterogeneous data but is not allowed to rewrite it. After the data is successfully submitted, other systems can synchronize the latest heterogeneous resource data from the heterogeneous device through the process of data synchronization, that is, the processed target mirror heterogeneous resources mentioned above.

[0059] Through this embodiment, the first system can send the processed target mirror heterogeneous resources to the heterogeneous device, and the heterogeneous device stores the processed target mirror heterogeneous resources, enabling other systems to synchronize the processed target mirror heterogeneous resources, thereby improving the consistency of heterogeneous resources among systems.

[0060] In one embodiment, before synchronizing the above-mentioned target mirror heterogeneous resources to the mirror storage space in the first system, it further includes: obtaining the occupancy status corresponding to the above-mentioned target mirror heterogeneous resources; if it is detected that the occupancy status of the above-mentioned target mirror heterogeneous resources is unoccupied, returning an allow-synchronization message to the above-mentioned first system; the allow-synchronization message is used to instruct the above-mentioned first system to read the above-mentioned target mirror heterogeneous resources to the mirror storage space.

[0061] In this embodiment, when the first system synchronizes the target mirror heterogeneous resources, it can first obtain the occupancy of the target mirror heterogeneous resources and then perform the synchronization and subsequent processing of the target mirror heterogeneous resources. Among them, at the same time, only one system is allowed to occupy the target mirror heterogeneous resources.

[0062] When the first system needs to synchronize the target mirror heterogeneous resources, the heterogeneous device can obtain the current occupancy status corresponding to the above-mentioned target mirror heterogeneous resources. If the heterogeneous device detects that the occupancy status of the above-mentioned target mirror heterogeneous resources is unoccupied, the heterogeneous device can return an allow-synchronization message to the above-mentioned first system; thus, the first system can read the target mirror heterogeneous resources into the mirror storage space in the first system based on the allow-synchronization message.

[0063] Specifically, taking the example that the first system needs to synchronize the target mirror heterogeneous resources, the first system can map the heterogeneous resources to the current process address space, so that the mapped heterogeneous resources can be used as heterogeneous resources called by other systems. Among them, the mapped heterogeneous resources are mainly local mirror memory. For the scenario where the heterogeneous resources and the operating system are on the same device, the first system can also directly map the physical pages corresponding to the heterogeneous resources.

[0064] For the target mirror heterogeneous resources that need to be called, the first system can obtain the ownership permission of the target mirror heterogeneous resources, that is, obtain the occupancy status of the target mirror heterogeneous resources. Among them, the owner refers to the owner of the heterogeneous resources. Taking the operating system as the granularity, non-owner systems can read heterogeneous data but are not allowed to rewrite it.

[0065] When the first system wants to obtain the ownership permission of the heterogeneous resources, it first sends a message to obtain the ownership permission to the heterogeneous device through the heterogeneous resource abstraction layer via the FPGA / network card. Among them, via the FPGA / network card, when the heterogeneous resources and the operating system are deployed on the same device, the transmission can be carried out without passing through the FPGA / network card. This description also applies to subsequent processes such as releasing the ownership permission, synchronizing heterogeneous data, and submitting heterogeneous data.

[0066] After receiving the message, the heterogeneous device first checks whether there is any other system holding the target mirror heterogeneous resources in this interval. If there is no conflict, it records the relevant information and returns success; otherwise, it returns failure or blocks and waits until the holder releases the corresponding ownership permission. Thus, after the first system obtains the ownership permission of the target mirror heterogeneous resources, it can determine that the first system occupies the target mirror heterogeneous resources, and the first system can process the target mirror heterogeneous resources.

[0067] Through this embodiment, the heterogeneous device can check the occupancy status of the target mirror heterogeneous resources based on the synchronization request of the first system for the target mirror heterogeneous resources, and change the owner of the target mirror heterogeneous resources to the above-mentioned first system when it is unoccupied, so that only the first system can process the target mirror heterogeneous resources during the period when the first system is the owner, improving the data integrity and consistency when processing heterogeneous data.

[0068] In one embodiment, after synchronizing the above-mentioned target mirror heterogeneous resources to the mirror storage space in the above-mentioned first system, the following steps are further included: obtaining an occupancy release instruction sent by the above-mentioned first system; according to the above-mentioned occupancy release instruction, releasing the occupancy of the above-mentioned first system on the above-mentioned target mirror heterogeneous resources; and updating the occupancy status of the above-mentioned target mirror heterogeneous resources according to the current call request corresponding to the above-mentioned target mirror heterogeneous resources.

[0069] In this embodiment, after the first system processes the target mirror heterogeneous resources, it is necessary to promptly release the occupancy of the target mirror heterogeneous resources. For example, the first system can send an occupancy release instruction for the target mirror heterogeneous resources to the heterogeneous device, so that the heterogeneous device releases the occupancy of the target mirror heterogeneous resources by the first system according to the occupancy release instruction. Among them, during the period when the target mirror heterogeneous resources are occupied by the first system, there may be other systems that want to call the target mirror heterogeneous resources, then these systems can queue up. When the occupancy of the target mirror heterogeneous resources is released, the heterogeneous device can update the occupancy status of the target mirror heterogeneous resources according to the current call request corresponding to the target mirror heterogeneous resources. For example, update it to be occupied by some other system.

[0070] Specifically, the above-mentioned occupancy status can be represented by the owner. The system that occupies the target mirror heterogeneous resources can be called the owner. Then the heterogeneous device can perform owner migration based on the call requests of each system for the target mirror heterogeneous resources. Among them, owner migration includes obtaining owner permissions and releasing owner permissions. After obtaining the owner permissions, data modification operations can be performed. During this period, if other systems want to obtain the owner permissions, they need to queue up and wait until the holder returns the owner permissions. Among them, the obtaining and releasing of owner permissions are in units of heterogeneous resource intervals, that is, multiple systems are allowed to obtain the owner permissions of different intervals of heterogeneous resources simultaneously.

[0071] Among them, taking the first system's call of the target mirror heterogeneous resources as an example, when the first system finishes processing the target mirror heterogeneous resources, it should immediately release the held resource owner permissions. At this time, if there are other systems waiting for the owner permissions, the heterogeneous device wakes up the waiters and updates the recorded owner information.

[0072] Through this embodiment, the heterogeneous device changes the occupancy status of the target mirror heterogeneous resources based on the calls of each system for the target mirror heterogeneous resources, thereby improving the data integrity and consistency when processing heterogeneous data.

[0073] In an exemplary embodiment, as Figure 3 shown, Figure 3 is a flowchart of a method for processing heterogeneous resources among multiple systems in another embodiment. This embodiment includes:

[0074] Heterogeneous resources refer to device resources of other systems that exist in multiple systems and cannot be directly accessed by one system, such as memory information on other systems.

[0075] Heterogeneous resource abstraction is mainly used to shield the differences in underlying hardware and provide a unified heterogeneous resource read and write access interface for the upper layer. The underlying hardware includes but is not limited to customized FPGA links, direct-connected network card devices, etc., as well as other hardware / bus devices that can transmit data between multiple systems.

[0076] Heterogeneous address space is used to globally address heterogeneous resources among multiple systems to ensure that different systems see consistent heterogeneous address space, including heterogeneous resource addressing, virtual addresses mapped to kernel state, and virtual addresses mapped to user state. In addition, through the heterogeneous address space, the information of mapped heterogeneous resources on the system is also maintained, including which processes map which heterogeneous resources, and the corresponding mirror memory pages and other information.

[0077] Heterogeneous resource mapping refers to mapping the mirror memory of locally maintained heterogeneous resources to the kernel or user address space by establishing a page table. After the mapping is successful, the mirror memory can be directly operated by reading and writing.

[0078] Heterogeneous owner migration, where the heterogeneous owner refers to the owner of heterogeneous resources, with the operating system as the granularity, and only systems with owner permissions are allowed to modify heterogeneous resources. Heterogeneous owner migration refers to obtaining or releasing owner permissions for heterogeneous resources. Systems that do not obtain owner permissions will return failure according to the policy, or block and wait until the holder releases the owner permissions. The acquisition and release of owner permissions are based on heterogeneous resource intervals, that is, multiple systems are allowed to obtain heterogeneous resource owner permissions in different intervals at the same time.

[0079] Heterogeneous data synchronization refers to synchronizing heterogeneous resource data to the mirror memory of the own system, or submitting the mirror memory data of the system to heterogeneous resource devices. The former has no owner authority control, while the latter requires a system with owner authority to allow execution.

[0080] User-mode API refers to heterogeneous resource-related interfaces provided in user mode. It includes heterogeneous resource mapping, heterogeneous owner migration, and heterogeneous data synchronization. Among them, heterogeneous resource mapping, heterogeneous owner migration, and heterogeneous data synchronization are all interfaces in the current Linux system, which have functions related to managing heterogeneous resources. At the same time, API interfaces that implement similar functions mentioned above by adding new system calls and other methods should also be regarded as the scope of protection of the present invention.

[0081] This embodiment includes the following steps:

[0082] Step 301: Map heterogeneous resources to the current process address space.

[0083] Among them, the mapped heterogeneous resources are mainly the local mirror memory of the system. For the scenario where the heterogeneous resources and the running system are on the same device, the physical pages corresponding to the heterogeneous resources can also be directly mapped.

[0084] Step 302: Obtain the owner permission of the heterogeneous resources.

[0085] Among them, the system where the heterogeneous resources are located maintains the owner information of the heterogeneous resources. When the system wants to obtain the owner permission of the heterogeneous resources, it first sends a message to obtain the owner permission to the heterogeneous device through the heterogeneous resource abstraction layer via the FPGA / network card. After receiving the message, the heterogeneous device first checks whether there is any other system holding the heterogeneous resources in this interval. If there is no conflict, it records the relevant information and returns success; otherwise, it returns failure or blocks and waits until the holder releases the corresponding owner permission, as shown in Figure 3 Step 312 in

[0086] Step 303: Synchronize the heterogeneous data to the local mirror memory.

[0087] Among them, the system first obtains the address information of the heterogeneous resources to be synchronized from the heterogeneous address space, and then calls the read interface provided by the heterogeneous resource abstraction layer to read the heterogeneous resource data from the heterogeneous device into the local mirror memory. Among them, during the process of writing the heterogeneous data into the local mirror memory, the heterogeneous device supports enabling concurrent protection to prevent other threads in this system from reading incomplete data.

[0088] Step 304: Read and write the heterogeneous data.

[0089] Among them, the system performs custom read and write operations on the heterogeneous data according to the business processing logic.

[0090] Step 305: Submit the modified data to the heterogeneous device.

[0091] Among them, since the heterogeneous data read and written in the previous step is actually a piece of mirror memory, after the modification is completed, it is also necessary to submit the modification to the heterogeneous device. The system first obtains the address information of the heterogeneous resources to be submitted from the heterogeneous address space, and then calls the write interface provided by the heterogeneous resource abstraction layer to write the data in the local mirror memory into the heterogeneous device. Among them, during the process of writing the heterogeneous data into the heterogeneous device, concurrent protection is supported to prevent other systems from reading incomplete data. If the system submitting the above heterogeneous resources does not have the owner permission, this submission operation will return failure.

[0092] After the data submission is successful, other systems can synchronize the latest heterogeneous resource data by executing Step 303.

[0093] Step 306: Release the owner permission.

[0094] Among them, when the heterogeneous resource data modification is completed, the system should immediately release the resource owner permission it holds. At this time, if there is another system waiting for this owner permission, wake up the waiters and update the recorded owner information, as shown in step 312.

[0095] Through the above embodiments, the heterogeneous device constructs a heterogeneous device connected to each system, manages the mirror heterogeneous resources of each system through the heterogeneous device, and each system can call and synchronize the mirror heterogeneous resources between different systems by combining the virtual address through the heterogeneous device, achieving the technical effect of improving the processing efficiency of heterogeneous resources among multiple systems. Moreover, through heterogeneous space management and resource mapping, the heterogeneous resources that cannot be directly read and written are indirectly mapped to the current system, and combined with functions such as heterogeneous owner migration and data synchronization, the cross-system sharing characteristics of heterogeneous resources are realized.

[0096] Compared with the current solution of transmitting data through network packets, the present application allows direct sharing of read-write heterogeneous resources among multiple systems, just like in the same process address space, with advantages such as high collaboration efficiency, flexibility, and ease of use.

[0097] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are displayed in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or steps in other steps.

[0098] Based on the same inventive concept, the embodiments of the present application also provide a heterogeneous resource processing device among multiple systems for implementing the above-mentioned heterogeneous resource processing method among multiple systems. The solution provided by this device to solve the problem is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the heterogeneous resource processing device among multiple systems provided below can refer to the limitations on the heterogeneous resource processing method among multiple systems in the above text, and will not be repeated here.

[0099] In an exemplary embodiment, as Figure 4 shown, a heterogeneous resource processing device among multiple systems is provided, including: an acquisition module 500, a determination module 502, and a processing module 504, where:

[0100] An acquisition module 500 is configured to acquire a target virtual address corresponding to a target heterogeneous resource in a second system sent by a first system; the heterogeneous device is respectively connected to the first system and the second system through a heterogeneous resource interface; the heterogeneous device stores one or more mirrored heterogeneous resources; each of the mirrored heterogeneous resources belongs to the first system or the second system.

[0101] A determination module 502 is configured to determine a target mirrored heterogeneous resource corresponding to the target heterogeneous resource according to the target virtual address.

[0102] A processing module 504 is configured to synchronize the target mirrored heterogeneous resource to a mirrored storage space in the first system; the first system is configured to process the target mirrored heterogeneous resource in the mirrored storage space to obtain a corresponding processing result.

[0103] In one embodiment, the apparatus further includes: a mapping module, configured to acquire respective virtual addresses corresponding to respective heterogeneous resources sent by the first system and / or the second system; the respective virtual addresses are within a preset virtual address range; map the respective mirrored heterogeneous resources corresponding to the respective heterogeneous resources to the respective virtual addresses to obtain a mapping relationship between the respective virtual addresses and the respective mirrored heterogeneous resources.

[0104] In one embodiment, the determination module 502 is configured to determine a starting call address and an ending call address according to the target virtual address; determine a called interval from the mirrored heterogeneous resources corresponding to the target heterogeneous resource according to the starting call address and the ending call address, and use the called interval as the target mirrored heterogeneous resource.

[0105] In one embodiment, the apparatus further includes: a synchronization module, configured to acquire the processed target mirrored heterogeneous resource sent by the first system according to the processing result; store the processed target mirrored heterogeneous resource; the processed target mirrored heterogeneous resource is used to instruct the second system to synchronize the processed target mirrored heterogeneous resource.

[0106] In one embodiment, the apparatus further includes: a detection module, configured to acquire an occupancy status corresponding to the target mirrored heterogeneous resource; if it is detected that the occupancy status of the target mirrored heterogeneous resource is unoccupied, return an allow-synchronization message to the first system; the allow-synchronization message is used to instruct the first system to read the target mirrored heterogeneous resource to the mirrored storage space.

[0107] In one embodiment, the above-mentioned device further includes: a release module, configured to obtain an occupancy release instruction sent by the above-mentioned first system; release the occupancy of the above-mentioned target mirror heterogeneous resource by the above-mentioned first system according to the above-mentioned occupancy release instruction; and update the occupancy status of the above-mentioned target mirror heterogeneous resource according to the current call request corresponding to the above-mentioned target mirror heterogeneous resource.

[0108] Each module in the above-mentioned heterogeneous resource processing device among multiple systems can be implemented in whole or in part by software, hardware, and their combination. Each of the above-mentioned modules can be embedded in the processor in the computer device in hardware form or be independent of it, or can be stored in the memory in the computer device in software form, so as to facilitate the processor to call and execute the operations corresponding to each of the above-mentioned modules.

[0109] In an exemplary embodiment, a computer device is provided. The computer device can be a heterogeneous device, and its internal structure diagram can be as Figure 5 shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store heterogeneous resource data. The input / output interface of the computer device is used for exchanging information between the processor and external devices. The communication interface of the computer device is used for communicating with external terminals through a network connection. When the computer program is executed by the processor, it implements a method for processing heterogeneous resources among multiple systems.

[0110] Those skilled in the art can understand that Figure 5 the structure shown in

[0111] is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0112] In one embodiment, when the processor executes the computer program, it also implements the above-mentioned method for processing heterogeneous resources among multiple systems.

[0113] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the heterogeneous resource processing method between multiple systems described above is implemented.

[0114] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the heterogeneous resource processing method between multiple systems described above is implemented.

[0115] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0116] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, artificial intelligence (AI) processors, etc., without limitation.

[0117] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in the present application.

[0118] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation to the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A method for processing heterogeneous resources among multiple systems, characterized in that: Applied to heterogeneous devices, the method includes: Obtaining a target virtual address corresponding to a target heterogeneous resource in a second system sent by the first system; the heterogeneous device is connected to the first system and the second system respectively through a heterogeneous resource interface; the heterogeneous device stores one or more mirrored heterogeneous resources; each of the mirrored heterogeneous resources belongs to the first system or the second system; Determine, according to the target virtual address, a target mirror heterogeneous resource corresponding to the target heterogeneous resource; The target mirror heterogeneous resource is synchronized to the mirror storage space in the first system; the first system is used to process the target mirror heterogeneous resource in the mirror storage space to obtain a corresponding processing result.

2. The method according to claim 1, characterized in that Before obtaining the target virtual address corresponding to the target heterogeneous resource in the second system sent by the first system, the method further includes: Acquire each virtual address corresponding to each heterogeneous resource sent by the first system and / or the second system; each of the virtual addresses is within a preset virtual address range; Each mirrored heterogeneous resource corresponding to each of the heterogeneous resources is mapped to the corresponding virtual address to obtain a mapping relationship between each of the virtual addresses and each of the mirrored heterogeneous resources.

3. The method according to claim 1, characterized in that The step of determining the target mirror heterogeneous resource corresponding to the target heterogeneous resource according to the target virtual address includes: Determine the start call address and the end call address according to the target virtual address; According to the start calling address and the end calling address, a called interval is determined from the mirrored heterogeneous resource corresponding to the target heterogeneous resource as the target mirrored heterogeneous resource.

4. The method according to claim 1, characterized in that: After synchronizing the target mirror heterogeneous resource to the mirror storage space in the first system, the method further includes: Obtaining a processed target image heterogeneous resource sent by the first system according to the processing result; The processed target mirror heterogeneous resources are stored; the processed target mirror heterogeneous resources are used to instruct the second system to synchronize the processed target mirror heterogeneous resources.

5. The method according to any one of claims 1 to 4, characterized in that: Before synchronizing the target mirror heterogeneous resource to the mirror storage space in the first system, the method further includes: Obtaining the occupancy status of the target image heterogeneous resources; If it is detected that the occupancy state of the target mirror heterogeneous resource is not occupied, synchronization permission information is returned to the first system; the synchronization permission information is used to instruct the first system to read the target mirror heterogeneous resource into the mirror storage space.

6. The method according to any one of claims 1 to 4, characterized in that: After synchronizing the target mirror heterogeneous resource to the mirror storage space in the first system, the method further includes: Obtaining an occupancy release instruction sent by the first system; According to the occupation release instruction, the first system releases the occupation of the target image heterogeneous resource; According to the call request currently corresponding to the target image heterogeneous resource, the occupation status of the target image heterogeneous resource is updated.

7. A heterogeneous resource processing device among multiple systems, characterized in that: Applied to heterogeneous equipment, the device comprises: An acquisition module is used to acquire a target virtual address corresponding to a target heterogeneous resource in a second system sent by the first system; the heterogeneous device is connected to the first system and the second system respectively through a heterogeneous resource interface; the heterogeneous device stores one or more mirrored heterogeneous resources; each of the mirrored heterogeneous resources belongs to the first system or the second system; A determination module, used to determine a target mirror heterogeneous resource corresponding to the target heterogeneous resource according to the target virtual address; The processing module is used to synchronize the target mirror heterogeneous resources to the mirror storage space in the first system; the first system is used to process the target mirror heterogeneous resources in the mirror storage space to obtain corresponding processing results.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images