Data transmission method and apparatus, storage medium, and electronic device

Abstract

The application provides a data transmission method and device, a storage medium and an electronic device. The method comprises the following steps: receiving a service data update request of a transaction node, and obtaining corresponding service data; allocating a memory address space for the service data, and storing the service data in the memory of a management node according to address information of the memory address space; writing the service data stored in the memory of the management node and additional information of the service data into a cache area of the management node, obtaining cache data to be transmitted; and sending the cache data to the transaction node, so that the transaction node stores the service data in the memory of the transaction node according to the address information of the memory address space, and writes the service data stored in the memory of the transaction node into a cache area of the transaction node, so as to facilitate a transaction process in the transaction node to call the service data. The method provided in the application can support a node to process complex objects of any size and any type.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a data transmission method and device, storage medium and electronic equipment. BACKGROUND

[0002] With the vigorous development of the Internet, especially the mobile Internet, the high concurrency and high performance requirements of e-commerce background system for large traffic are higher and higher. In such a background system, it is often necessary to read the database or request to call the basic support service at a high frequency in a short time to obtain a large amount of complex business data.

[0003] Therefore, in some scenarios that do not require strict consistency and complete accuracy, cache technology is often used as a buffer between the transaction system and the data source (database / basic service). The cache can make the transaction system get data faster, and at the same time, resist most of the pressure impact for the underlying data source. This is a common way in the industry, and its implementation includes simple and easy-to-use in-memory database services such as Redis, Memcached, or embedded databases such as Berkeley DB, Level DB, and further pure memory embedded databases such as LMDB. However, the prior art cannot support complex objects, nor can it support data processing of large-scale distributed cache clusters. SUMMARY

[0004] The technical problem to be solved by the present application is to provide a data transmission method that can support nodes to process complex objects of any size and any type.

[0005] The present application also provides a data transmission device to ensure the implementation and application of the above method in practice.

[0006] A data transmission method, comprising:

[0007] In the case of receiving a preset transaction node business data update request, obtaining the business data corresponding to the business data update request;

[0008] Allocating a memory address space for the business data, and storing the business data in the memory of the management node according to the address information of the memory address space;

[0009] Write the business data stored in the memory of the management node and the additional information of the business data into the cache area of the management node to obtain the cache data to be transmitted, and the additional information at least includes the address information of the memory address space;

[0010] transmit the cache data to the transaction node, so that the transaction node stores the service data into the memory of the transaction node according to the address information of the memory address space, and writes the service data that has been stored into the memory of the transaction node into the cache area of the transaction node, to facilitate a transaction process in the transaction node to call the service data.

[0011] A data transmission apparatus comprises:

[0012] a receiving unit configured to, in response to receiving a preset service data update request of a transaction node, acquire service data corresponding to the service data update request;

[0013] an executing unit configured to allocate a memory address space for the service data, and store the service data into the memory of the management node according to address information of the memory address space;

[0014] a writing unit configured to write the service data that has been stored into the memory of the management node and additional information of the service data into a cache area of the management node, obtain cache data to be transmitted, and the additional information at least includes the address information of the memory address space;

[0015] a transmitting unit configured to transmit the cache data to the transaction node, so that the transaction node stores the service data into the memory of the transaction node according to the address information of the memory address space, and writes the service data that has been stored into the memory of the transaction node into the cache area of the transaction node, to facilitate a transaction process in the transaction node to call the service data.

[0016] A storage medium comprises a storage instruction, wherein when the instruction is executed, the device where the storage medium is located performs the data transmission method as described above.

[0017] An electronic device comprises a memory, and one or more instructions, wherein the one or more instructions are stored in the memory and configured to be executed by one or more processors to perform the data transmission method as described above.

[0018] Based on the above, the application provides a data transmission method and device, a storage medium and an electronic device. The method comprises the following steps: in the case that a preset transaction node receives a service data update request, obtaining service data corresponding to the service data update request; allocating a memory address space to the service data, and storing the service data in the memory of the management node according to the address information of the memory address space; writing the service data stored in the memory of the management node and additional information of the service data into the cache area of the management node to obtain cache data to be transmitted, wherein the additional information at least comprises the address information of the memory address space; sending the cache data to the transaction node, so that the transaction node stores the service data in the memory of the transaction node according to the address information of the memory address space, and writes the service data stored in the memory of the transaction node into the cache area of the transaction node, so as to facilitate the transaction process in the transaction node to call the service data. The method can support the node to process complex objects of any size and any type, and can support large-scale cluster data processing. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of the provided drawings.

[0020] Figure 1 A method flowchart of a data transmission method provided by the present application;

[0021] Figure 2 A method flowchart of another data transmission method provided by the present application;

[0022] Figure 3 An implementation scenario example diagram provided by the present application;

[0023] Figure 4 A method flowchart of another data transmission method provided by the present application;

[0024] Figure 5 A management mechanism example diagram provided by the present application;

[0025] Figure 6 A distributed scenario cooling time schematic diagram provided by the present application;

[0026] Figure 7 A structure schematic diagram of a data transmission device provided by the present application;

[0027] Figure 8 A structural schematic diagram of an electronic device provided by the present application. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0029] In the present application, the terms “comprising”, “containing” or any other variants thereof are intended to cover non-exclusive containing, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the sentence “including a…” does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0030] The embodiment of the present application provides a data transmission method, which can be applied to a management node. The management node can be various types of electronic devices. In some application scenarios, the management node can be a server in a cluster. The cluster further includes at least one transaction node. Each transaction node in the cluster runs at least one transaction process. The management node and each transaction node are in the same multicast network. The method flowchart of the data transmission method is as shown in Figure 1 The embodiment of the present application provides a data transmission method, which can be applied to a management node. The management node can be various types of electronic devices. In some application scenarios, the management node can be a server in a cluster. The cluster further includes at least one transaction node. Each transaction node in the cluster runs at least one transaction process. The management node and each transaction node are in the same multicast network. The method flowchart of the data transmission method is as shown in

[0031] S101: In the case that a preset transaction node service data update request is received, service data corresponding to the service data update request is acquired.

[0032] In the embodiment, the number of service data update requests can be one or more. The service data update request can be sent by the transaction node in the case that the transaction node does not have data required by the current transaction service and not expired in the cache.

[0033] Optionally, after the management node receives the data update request, the service data corresponding to the service data update request can be acquired first. The service data can be data required by the current transaction service of the transaction node.

[0034] S102: Memory address space is allocated for the service data, and the service data is stored into the memory of the management node according to address information of the memory address space.

[0035] In the embodiment, the memory address space can be a continuous virtual address space in a memory address space interval to be allocated in the management node.

[0036] Optionally, the memory address space can be used by an operating system of the management node to allocate memory for the service data, and the management node writes the service data into the memory allocated by the operating system.

[0037] S103: write the service data stored in the memory of the management node and additional information of the service data into a cache area of the management node, to obtain cache data to be transmitted, the additional information at least including address information of the memory address space.

[0038] In the embodiment, the additional information can further include at least one of an expiration time of the service data, a first invalidation cooling time of the service data in the management node, and a second invalidation cooling time of the service data in the transaction node.

[0039] S104: send the cache data to the transaction node, so that the transaction node stores the service data into a memory of the transaction node according to the address information of the memory address space, and writes the service data stored in the memory of the transaction node into a cache area of the transaction node, to facilitate a transaction process in the transaction node to call the service data.

[0040] In the embodiment, the cache data can be sent to the transaction node through a multicast network.

[0041] The method provided by the embodiment of the application can support the nodes to process complex objects of any size and any type, and can support data processing of a large-scale cluster.

[0042] In an embodiment provided by the application, based on the above-described implementation process, optionally, the memory address space is allocated for the service data, such as Figure 2 as shown, including:

[0043] S201: determine a data amount of the service data.

[0044] In the embodiment, the service data can be data of any data amount size, for example, data of 10 KB, 1 MB or 1 GB, etc.

[0045] S202: allocate a continuous memory address space for the service data by a preset address space manager based on the data amount; a capacity of the memory address space allocated for the service data is greater than or equal to the data amount of the service data.

[0046] In the embodiment, the address space manager can manage the virtual address space, and can group, segment and mark a whole address space. The address space can be allocated according to the data size of the service data. For example, when the data size of the service data is 10 KB, a continuous memory address space with a capacity of at least 10 KB can be divided from a virtual address space allocation pool, and the memory address space is locked when being allocated. The virtual address space allocation pool can be a memory address space interval.

[0047] In an embodiment provided by the application, based on the above implementation process, the storing the service data into the memory of the management node according to the address information of the memory address space can include:

[0048] applying, to an operating system of the management node, a memory matching the capacity of the memory address space based on the address information of the memory address space, wherein the first address of the memory matches the address information of the memory address space;

[0049] storing the service data into the memory.

[0050] In the embodiment, the address information of the memory address space can be a relative address. The memory matching the capacity of the memory address space can be applied to the management node to obtain the memory allocated by the operating system, and the capacity of the memory matches the capacity of the memory address space.

[0051] In an embodiment provided by the application, based on the above implementation process, after the writing the service data stored in the memory of the management node and the additional information of the service data into the cache area of the management node, the method further includes:

[0052] cooling the cache area of the management node when reaching a preset service data expiration time, to avoid new access to the cache area;

[0053] releasing the service data and the additional information in the cache area of the management node and recycling the memory address space when reaching a preset first invalid cache cooling time;

[0054] The first invalid cache cooling time is greater than a second invalid cache cooling time of the service data in the transaction node, the second invalid cache cooling time is greater than a transaction longest allowed time of a transaction process in the transaction node, and the second invalid cache cooling time refers to a cooling time of the service data in the transaction node.

[0055] In the embodiment, a possible way to cool the cache area of the management node is to remove the cache area from the cache list of the management node and place it in a cooling pool, and after the first invalid cache cooling time is reached, the address space is returned to the address space manager and the data is erased, and the memory is returned to the operating system.

[0056] Optionally, the first invalid cache cooling time is greater than the second invalid cache cooling time, which can avoid the influence of network transmission and instantaneous interruption. Since there may be a transaction process using the cache in the transaction node during the exit process, the second invalid cache cooling time is set to be greater than the longest allowed transaction time, so that the transaction process can complete the transaction processing, and then the memory address space in the transaction node is recycled. After the memory address space in the transaction node is recycled, the memory address space of the management node is recycled.

[0057] In an embodiment provided by the application, based on the above-mentioned implementation process, optionally, the obtaining of the business data corresponding to the business data update request comprises:

[0058] Obtaining the business data identifier contained in the business data update request;

[0059] According to the business data identifier, a preset external system is called to obtain the business data corresponding to the business data update request.

[0060] In the embodiment, the business data required by the transaction node can be obtained from the external system based on the business data identifier.

[0061] Referring to Figure 3 An embodiment provided by the application provides an implementation scene example diagram, Figure 3 The implementation scene shown in the figure can be a distributed embedded cache cluster system MMCache comprising a cluster cache management node Server and a plurality of distributed transaction nodes Node. The cluster cache management node is the management node described above, and the distributed transaction node is the transaction node described above. The cluster cache management node and the distributed transaction node can be servers in the cluster.

[0062] In the embodiment, referring to Figure 4The method flow chart of another data transmission method provided by the embodiment of the application is shown in FIG. 4. The Node is responsible for using the cache data and reporting the cache invalidation information to the Server. Specifically, in the transaction process of the Node, when certain service data is needed, the Node searches the cache for the data. If the service data is not found, an update request is generated. The cache management process in the Node collects the update requests converged by the transaction processes and reports to the Server. Meanwhile, the Node is responsible for writing the high-frequency update messages in the cluster multicast to the local cache. The transaction process in the Node can directly use the cache data in the form of process memory. The Nodes are completely independent and have no awareness of each other.

[0063] In the embodiment, the Server is responsible for interacting with the external database or data providing system and generating, issuing and managing the cache data. Specifically, the cache management process in the Server collects the update requests converged by the Nodes, is responsible for data exchange with the external system, writes the updated data to the cache and updates to the Nodes through the multicast network.

[0064] Specifically, in the whole MMCache working scenario, three kinds of management are needed.

[0065] 1. Address space management of the MMCache cache, in order to recycle the pre-specified address space for allocation of any size.

[0066] 2. Node memory management, which does not need to be concerned by the application and depends on the operating system, which will not be described in detail.

[0067] 3. Management of the state of the MMCache cache itself, such as the real-time state, occupied address, memory condition, LRU list, etc.

[0068] Among them, refer to Figure 5 The management mechanism example diagram of the MMCache provided by the embodiment of the application is shown in FIG. 5. The Server is responsible for the cache management and synchronization of the whole cluster. Since the MMCache needs to support complex objects and any size among the clusters, the MMCache needs to perform relatively complex memory management, complete the management, application and allocation of the address space in the specified range address space, and the resources used by the cache will be recycled for a long time. Therefore, the management of the address space is needed to support the cache of any size. In the application, the Server uses a certain process virtual address space as an allocation pool according to the pre-set configuration to support the recycling of the cache of any size. Among them, the address space has almost no cost and the length is much larger than the actual memory. Here, the pre-set configuration can prevent the conflict with other databases or caches using the memory mapping technology.

[0069] Slab algorithm can be well qualified for memory resource management, in the present application, it is innovatively applied to virtual address space management, a whole block of address space is grouped, segmented and marked, and address space is allocated according to actual cache size, and unused space resources are quickly recycled, arranged and combined. In this design, when the Server needs to store cache, it will initiate an application request to the "address space manager"; get the address space for use; and when the cache is released after use, the address space is returned to the manager. Note that the Slab needs to be locked when allocating address space, but since this operation is only performed in the cache management process of each node, it will not cause write blocking.

[0070] The Server can store complex objects in the mapping memory and finally pass them to the Node in the form of memory blocks, which are then opened and used in the same way, ensuring that serialization and deserialization are not required throughout the process. Memory reading and writing of arbitrary objects are performed using the same method, which maximizes the reduction of cache cluster communication costs.

[0071] The exit mechanism of the cache deleted due to insufficient memory, cache expiration or other reasons is particularly important in a distributed cluster. This part uses differentiated cooling pool technology to ensure that the cache in the Server that is in the exit process is still available in the Node (possibly with a transaction process being used), and to ensure that the available space in the Server that has been recycled is also recycled in the Node. For each cache about to exit, two additional pieces of information are set: Server exit time and Node exit time. The Node exit time only needs to be longer than the maximum allowed time for a transaction, and the Server exit time needs to be long enough to offset the impact of network transmission, instantaneous interruption, etc. In the exit process, the relevant cache is first removed from the cache list to ensure that no new access is added, and is placed in the cooling pool for cooling. When the cache in the cooling pool reaches the exit time, the address space is returned to the space manager, the data is erased, and the memory is returned to the operating system.

[0072] In the present embodiment, the local area network reliable multicast technology is used as the internal protocol of the cluster for transmitting management control information and cache data streams. Using multicast can maximize the use of local area network bandwidth and enable adaptive transmission. The data propagation of the entire cluster is 1->n, and using this protocol can avoid the network bandwidth of the Server node becoming a bottleneck for cache update speed, enabling the cache transmission bandwidth of the entire cluster to reach the GB level. The reliability of transmission ensures that if an individual Node does not receive or receives incompletely, it can be configured to require retransmission, ignore or wait for the next cache update.

[0073] In the application layer in the embodiment, the memory mapping which has been dumped into a cache file is attached with service data additional data through a multicast technology, and the service data additional data mainly includes operations, virtual address spaces, expiration times, and deletion times; and the Server sends the memory mapping and the service data additional data to the Node, and the Node expands the memory mapping in place according to the virtual address spaces, and the cache data and internal pointers, complex structures, and containers are immediately available. This operation does not need to pass through any serialization and deserialization, is independent of the complexity of the cache file data, and is independent of the cache size, and can support extremely fast reading speed.

[0074] In an embodiment provided by the application, in order to better understand the technical solutions of the application, examples are used for illustration.

[0075] I. Usage scenarios and configuration presets

[0076] In a certain distributed service scenario, service A needs to call service B in a transaction to obtain certain data and relationships. The change frequency of the data and relationships is uncertain, and service A has a certain tolerance for data change lag. Service A has extremely high frequency and large amount of access to data and relationship information from B, but service B is very sensitive to pressure. This situation is more suitable for using MMCache in the cluster of service A.

[0077] First, ensure that the service A cluster is in the same local area network, and the network meets the multicast conditions. Second, ensure that each transaction node of service A is configured identically or similarly, and the operating system versions are similar, specifically in the same major version, to ensure consistent C++ memory layout.

[0078] Second, determine the granularity of the data and relationships provided by service B, that is, estimate the average size of the cache block, and estimate the total memory upper limit allowed by the cache framework and the pre-allocated address space interval based on this. For example: the size of a single cache is about 5KB, the total upper limit is 50GB, and it can accommodate millions of caches, and the size of the available address space interval is 4TB, the interval is 0x100000000000-0x140000000000, which is much larger than the actual memory. The address space management can be performed in a relatively loose environment. In the embodiment of the application, the expression method in the cache is not limited, and can include pointers, containers, etc., but the pointers are limited to the cache block (without complex cascade management).

[0079] Finally, the service A cluster needs to increase the Server node and enable the MMCache management service, and the MMCache management service is also enabled on the Node. The management service will start and occupy the 0x100000000000-0x140000000000 part in the process on all nodes. All nodes are configured in a multicast state.

[0080] II. Cache usage and update trigger.

[0081] Since the cache is implemented based on shared memory memory mapping, in each transaction node process of service A, the cache can be used as conveniently and quickly as using in-process memory. Assuming that in a transaction, data a and b are needed, where a has been cached in MMCache and has not expired or been deleted, the process can directly use it; b has not been cached or has been cached for too long and has expired, and needs to be obtained from service B. Then:

[0082] 1. The transaction process reports this to the management process of the node, and then suspends this segment of logic and starts the next step without blocking. If it cannot be suspended, it will keep checking and waiting.

[0083] 2. The node management process submits the update request to the Sever node, and the Sever node calls service B to get the latest data and relationship.

[0084] III. Cache writing and distribution management.

[0085] After Server gets the latest data b:

[0086] Step 1: According to the size of b (assuming 10KB here), apply for 10KB of address space to the "address space manager". The "address space manager" will provide at least 10KB of unused continuous space in the 0x100000000000-0x1400000000000 interval according to the Slab algorithm, and return the relative address ptr.

[0087] Step 2: Use absolute address memory mapping to apply for 10KB of memory to the operating system, and ensure that the first address is ptr.

[0088] Step 3: Load data b into ptr in its entirety, and if necessary, modify the container allcator in it to use memory only in ptr.

[0089] Step 4: Add additional information such as business expiration time and ptr address, and keep Server exit time and Node exit time empty (a value means that the resource needs to be released after a certain period of time).

[0090] Step 5: Hang ptr in the cache hash. Specifically, use lock-free technology to directly mount the new cache block at the head of the corresponding hash bucket through atomic operations.

[0091] Step 6: Package the cache block and transmit it to the Node through the multicast network, and be ready to respond to the retransmission request from the Node at any time.

[0092] IV. Management of cache reception and retrieval.

[0093] After the management process in the Node receives the new cached data completely, it repeats steps two through five executed by the Server on the Node, based on the PTR address, to write the cached data to the local cache. If there is incomplete data, it can choose to ignore and discard it or request it from the Server again. At this point, the transaction process waiting in the Node checks that the time of data 'b' has been updated and can read the content for subsequent business processing.

[0094] As long as the cache is accessed through MMCache, the local process can always trust the cache block that is accessed before its expiration time. Furthermore, each node and each process is independent and can access the cache at the speed of reading local memory.

[0095] V. Special Operations: Cache refresh and deletion process.

[0096] In this embodiment, refreshing and deleting cache blocks are both treated as adding cache blocks. The new cache block is inserted into the original cache list without locks. When deleting, a blank block is used. After the insertion operation, the original cache block can no longer be accessed by new transactions, so it will be removed from the cache list and placed in the cooling pool.

[0097] The above operation will be performed once on both the Server and the Node. However, the Server will simultaneously write the Server exit time: 300 seconds and the Node exit time: 60 seconds. The maximum transaction time for Service A is 30 seconds; the Server exit time is the first invalidation cache cooldown time mentioned above, and the Node exit time is the second invalidation cache cooldown time mentioned above.

[0098] 30 seconds later, as Figure 6 As shown, the transaction time is the shortest, so all transactions accessing the original cache (already in the cooling pool) on the Node have finished, meaning they are reclaimable. Since the access uses the most basic direct pointer, there is no indicator to check whether the access has ended; it can only be inferred from the time.

[0099] After 60 seconds, the Node exit time for this cache block expires. The Node removes the obsolete cache from the cooling pool, returns the address space to the manager, and releases the memory.

[0100] After 300 seconds, the server exit time for this cache block expires. The server removes the obsolete cache from the cooling pool, returns the address space to the manager, and releases the memory.

[0101] The technical solution provided by the embodiments of the present invention has the following advantages:

[0102] 1. Excellent read and write performance. MMCache is essentially read and write to local memory, which has reached the ultimate read and write performance, about 100 nanoseconds. Its read efficiency does not decrease with the increase of concurrency, nor does it decrease with the increase of data complexity. It can be widely used in read-only, read-write and high-performance scenarios. MMCache has reached the same cost as using cache and using process memory.

[0103] 2. Support for complex objects. Support for accessing any type of cache. MMCache can support any existing type, STL container, custom type and their mutual nesting and combination. This feature allows sharing of complex relationships / objects at the cluster level in the cache, greatly expanding the capabilities of the cache cluster.

[0104] 3. Support for fast updates. Any cache update request from a node will trigger an update of the entire cache cluster, and data can be quickly updated to all nodes in the cluster through multicast networks without considering bandwidth. The update speed will increase with the increase of cluster size and system pressure.

[0105] 4. Support for any size. MMCache has a mechanism to manage address space, so it can support any size of cache unit to be written, distributed and managed. This feature allows MMCache to run for a long time without worrying about pre-set address space being unavailable due to too much fragmentation, or being wasted due to inability to support any size.

[0106] In the scheme provided by the embodiment of the application, such a cache is very suitable for some high-performance, high-concurrency scenarios that do not rely much on data consistency, require high-frequency reading of data or services, and have complex and varying sizes of data internal relationships. For example, it can be applied to the calculation of international and domestic freight rates and search systems.

[0107] With Figure 1 Corresponding to the method, the embodiment of the application further provides a data transmission device for implementing the method. Figure 1 The data transmission device provided by the embodiment of the application can be applied to a management node, and a structure diagram thereof is shown in Figure 7 The data transmission device specifically comprises:

[0108] A receiving unit 701, configured to acquire business data corresponding to a business data update request of a preset transaction node in a case where the business data update request is received;

[0109] An execution unit 702, configured to allocate an address space of a memory to the business data, and store the business data into the memory of the management node according to address information of the address space of the memory;

[0110] a writing unit 703, configured to write the service data stored in the memory of the management node and additional information of the service data into a cache area of the management node, to obtain cache data to be transmitted, the additional information at least including address information of the memory address space;

[0111] a transmitting unit 704, configured to transmit the cache data to the transaction node, so that the transaction node stores the service data into the memory of the transaction node according to the address information of the memory address space, and writes the service data stored in the memory of the transaction node into a cache area of the transaction node, to facilitate a transaction process in the transaction node to call the service data.

[0112] In an embodiment provided by the present application, based on the above scheme, the execution unit 702 optionally includes:

[0113] a determining sub-unit, configured to determine a data volume of the service data;

[0114] an allocating sub-unit, configured to allocate a continuous memory address space for the service data by a preset address space manager based on the data volume; a capacity of the memory address space allocated for the service data is greater than or equal to the data volume of the service data.

[0115] In an embodiment provided by the present application, based on the above scheme, the execution unit 702 optionally includes:

[0116] an executing sub-unit, configured to apply, to an operating system of the management node, a memory matching the capacity of the memory address space based on the address information of the memory address space, a first address of the memory being consistent with the address information of the memory address space;

[0117] a storing sub-unit, configured to store the service data into the memory applied.

[0118] In an embodiment provided by the present application, based on the above scheme, the data transmission apparatus further includes:

[0119] a cooling unit, configured to cool the cache area of the management node after a preset service data expiration time is reached, to avoid new access to the cache area;

[0120] a releasing unit, configured to release the service data and the additional information in the cache area of the management node and to recycle the memory address space after a preset first invalid cache cooling time is reached;

[0121] The first invalid cache cooling time is greater than a second invalid cache cooling time of the business data in the transaction node, the second invalid cache cooling time is greater than a transaction longest allowed time of a transaction process in the transaction node, and the second invalid cache cooling time refers to a time when the business data is cooled in the transaction node.

[0122] In an embodiment provided by the application, based on the above scheme, the receiving unit 701 is configured to:

[0123] Obtain the business data identifier included in the business data update request;

[0124] Call a preset external system according to the business data identifier to obtain the business data corresponding to the business data update request.

[0125] The specific principles and execution processes of each unit and module in the data transmission device disclosed in the embodiments of the application are the same as those of the data transmission method disclosed in the embodiments of the application, and can be referred to the corresponding part of the data transmission method provided in the embodiments of the application.

[0126] The embodiments of the application further provide a storage medium including stored instructions, wherein when the instructions are executed, the device where the storage medium is located performs the data transmission method, and the method specifically includes:

[0127] In the case that the business data update request of the preset transaction node is received, obtain the business data corresponding to the business data update request;

[0128] Allocate a memory address space for the business data, and store the business data in the memory of the management node according to the address information of the memory address space;

[0129] Write the business data stored in the memory of the management node and additional information of the business data into the cache area of the management node to obtain cache data to be transmitted, and the additional information at least includes the address information of the memory address space;

[0130] Send the cache data to the transaction node, so that the transaction node stores the business data into the memory of the transaction node according to the address information of the memory address space, and writes the business data stored in the memory of the transaction node into the cache area of the transaction node, so as to facilitate the transaction process in the transaction node to call the business data.

[0131] The above method, optionally, the memory address space allocated for the business data includes:

[0132] determining a data volume of the service data;

[0133] allocating a continuous memory address space for the service data based on the data volume by a preset address space manager; a capacity of the memory address space allocated for the service data is greater than or equal to the data volume of the service data.

[0134] The method described above, optionally, the method of storing the service data into the memory of the management node according to the address information of the memory address space, comprises:

[0135] applying, based on the address information of the memory address space, a memory matching the capacity of the memory address space to an operating system of the management node, a first address of the memory being consistent with the address information of the memory address space;

[0136] storing the service data into the memory applied.

[0137] The method described above, optionally, after the method of writing the service data stored in the memory of the management node and the additional information of the service data into the cache area of the management node, further comprises:

[0138] cooling the cache area of the management node after a preset service data expiration time is reached, so as to avoid new access to the cache area;

[0139] releasing the service data and the additional information in the cache area of the management node and recycling the memory address space after a preset first invalid cache cooling time is reached;

[0140] wherein the first invalid cache cooling time is greater than a second invalid cache cooling time of the service data in the transaction node, the second invalid cache cooling time is greater than a transaction longest allowed time of a transaction process in the transaction node, and the second invalid cache cooling time refers to a time when the service data is cooled in the transaction node.

[0141] The method described above, optionally, the method of obtaining the service data corresponding to the service data update request, comprises:

[0142] obtaining a service data identifier contained in the service data update request;

[0143] calling a preset external system according to the service data identifier to obtain the service data corresponding to the service data update request.

[0144] An electronic device is also provided by the embodiment of the application, and a structure diagram of the electronic device is as shown in Figure 8As shown, specifically includes a memory 801, and one or more instructions 802, wherein one or more instructions 802 stored in the memory 801, and configured to be executed by one or more processors 803 said one or more instructions 802 to operate as follows:

[0145] In the case of receiving a preset transaction node service data update request, obtaining the service data corresponding to the service data update request;

[0146] The memory address space is allocated for the service data, and the service data is stored in the memory of the management node according to the address information of the memory address space;

[0147] The service data stored in the memory of the management node and the additional information of the service data are written into the cache area of the management node, and the cache data to be transmitted is obtained, and the additional information at least includes the address information of the memory address space;

[0148] The cache data is sent to the transaction node, so that the transaction node stores the service data into the memory of the transaction node according to the address information of the memory address space, and the service data stored in the memory of the transaction node is written into the cache area of the transaction node, so that the transaction process in the transaction node calls the service data.

[0149] According to one or more embodiments of the present disclosure, Figure 1 Embodiments of the present disclosure provide a data transmission method, the method comprising:

[0150] In the case of receiving a preset transaction node service data update request, obtaining the service data corresponding to the service data update request;

[0151] The memory address space is allocated for the service data, and the service data is stored in the memory of the management node according to the address information of the memory address space;

[0152] The service data stored in the memory of the management node and the additional information of the service data are written into the cache area of the management node, and the cache data to be transmitted is obtained, and the additional information at least includes the address information of the memory address space;

[0153] The cache data is sent to the transaction node, so that the transaction node stores the service data into the memory of the transaction node according to the address information of the memory address space, and the service data stored in the memory of the transaction node is written into the cache area of the transaction node, so that the transaction process in the transaction node calls the service data.

[0154] Optionally, the method of allocating memory address space for the business data comprises:

[0155] Determining the data volume of the business data;

[0156] Allocating continuous memory address space for the business data based on the data volume by a preset address space manager; the capacity of the memory address space allocated for the business data is greater than or equal to the data volume of the business data.

[0157] Optionally, the method of storing the business data into the memory of the management node according to the address information of the memory address space comprises:

[0158] Applying memory matching the capacity of the memory address space to the operating system of the management node based on the address information of the memory address space; the first address of the memory is consistent with the address information of the memory address space;

[0159] Storing the business data into the memory applied.

[0160] Optionally, after the method of writing the business data stored in the memory of the management node and the additional information of the business data into the cache area of the management node, the method further comprises:

[0161] After reaching a preset business data expiration time, cooling the cache area of the management node to avoid new access to the cache area;

[0162] After reaching a preset first invalid cache cooling time, releasing the business data and the additional information in the cache area of the management node, and recycling the memory address space;

[0163] The first invalid cache cooling time is greater than a second invalid cache cooling time of the business data in the transaction node, and the second invalid cache cooling time is greater than the longest allowed transaction time of the transaction process in the transaction node. The second invalid cache cooling time refers to the time when the business data is cooled in the transaction node.

[0164] Optionally, the method of obtaining the business data corresponding to the business data update request comprises:

[0165] Obtaining the business data identifier contained in the business data update request;

[0166] Calling a preset external system according to the business data identifier to obtain the business data corresponding to the business data update request.

[0167] According to one or more embodiments of the present disclosure, Figure 7 Embodiments of the present disclosure provide a data transmission device, comprising:

[0168] a receiving unit configured to, in a case where a preset service data update request of a transaction node is received, acquire service data corresponding to the service data update request;

[0169] an executing unit configured to allocate a memory address space for the service data, and store the service data into a memory of the management node according to address information of the memory address space;

[0170] a writing unit configured to write the service data stored into the memory of the management node and additional information of the service data into a cache area of the management node, to obtain cache data to be transmitted, the additional information at least including the address information of the memory address space;

[0171] a transmitting unit configured to transmit the cache data to the transaction node, so that the transaction node stores the service data into a memory of the transaction node according to the address information of the memory address space, and writes the service data stored into the memory of the transaction node into a cache area of the transaction node, so as to facilitate a transaction process in the transaction node to call the service data.

[0172] In an embodiment provided by the present disclosure, based on the above scheme, optionally, the executing unit comprises:

[0173] a determining subunit configured to determine a data amount of the service data;

[0174] an allocating subunit configured to allocate a continuous memory address space for the service data by a preset address space manager based on the data amount; a capacity of the memory address space allocated for the service data is greater than or equal to the data amount of the service data.

[0175] In an embodiment provided by the present disclosure, based on the above scheme, optionally, the executing unit comprises:

[0176] an executing subunit configured to apply, to an operating system of the management node, a memory matching the capacity of the memory address space based on the address information of the memory address space, a first address of the memory being consistent with the address information of the memory address space;

[0177] a storing subunit configured to store the service data into the memory applied.

[0178] In an embodiment provided by the present application, based on the above scheme, optionally, the data transmission device further comprises:

[0179] a cooling unit, configured to cool the cache area of the management node after a preset service data expiration time is reached, so as to avoid new access to the cache area;

[0180] a releasing unit, configured to release the service data and the additional information in the cache area of the management node after a preset first invalid cache cooling time is reached, and recycle the memory address space;

[0181] wherein the first invalid cache cooling time is greater than a second invalid cache cooling time of the service data in the transaction node, the second invalid cache cooling time is greater than a transaction longest allowed time of a transaction process in the transaction node, and the second invalid cache cooling time refers to a time when the service data is cooled in the transaction node.

[0182] In an embodiment provided by the present application, based on the above scheme, optionally, the receiving unit is configured to:

[0183] obtain a service data identifier contained in the service data update request;

[0184] invoke a preset external system according to the service data identifier, and obtain service data corresponding to the service data update request.

[0185] It should be noted that each embodiment in the present specification adopts a progressive manner for description, and each embodiment mainly describes the difference from other embodiments, and the same and similar parts of each embodiment can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant part can be referred to the part of the method embodiment.

[0186] Finally, it should be noted that in this paper, relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0187] For the convenience of description, the above apparatus is described as various units respectively described in functions. Of course, the functions of the units can be implemented in one or more software and / or hardware when implementing the present application.

[0188] From the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software and the necessary general hardware platform. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which can be stored in a storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in various embodiments or some parts of the embodiments of the present application.

[0189] The above describes in detail the data transmission method provided by the present application, and the principle and implementation of the present application are described by applying specific examples. The above embodiment is only used to help understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation and application range will be changed; in summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A data transmission method, characterized in that, Applied to management nodes, including: Upon receiving a business data update request from a preset transaction node, obtain the business data identifier contained in the business data update request; and invoke a preset external system based on the business data identifier to obtain the business data corresponding to the business data update request. The process involves: determining the volume of the business data; allocating a contiguous memory address space for the business data based on the volume using a preset address space manager; the address space manager grouping, segmenting, and marking a whole address space, and allocating address space according to the volume of the business data; the capacity of the memory address space allocated to the business data being greater than or equal to the volume of the business data; requesting memory from the operating system of the management node that matches the capacity of the memory address space based on the address information of the memory address space, wherein the starting address of the memory is consistent with the address information of the memory address space; and storing the business data in the allocated memory. The service data already stored in the memory of the management node and the additional information of the service data are written into the cache area of ​​the management node to obtain the cached data to be transmitted. The additional information includes at least the address information of the memory address space. The cached data is sent to the transaction node, which then stores the business data in its memory according to the address information in the memory address space. The business data already stored in the transaction node's memory is then written into the transaction node's cache area, allowing the transaction process within the transaction node to access the business data. The management node and the transaction node are located in the same multicast network. The cached data is transmitted through the multicast network. After writing the business data and its additional information stored in the memory of the management node into the cache area of ​​the management node, the method further includes: cooling the cache area of ​​the management node after a preset business data expiration time is reached to avoid new access to the cache area; releasing the business data and its additional information in the cache area of ​​the management node and reclaiming the memory address space after a preset first invalidation cache cooldown time is reached; wherein, the first invalidation cache cooldown time is greater than the second invalidation cache cooldown time of the business data in the transaction node, the second invalidation cache cooldown time is greater than the maximum allowed transaction time of the transaction process in the transaction node, and the second invalidation cache cooldown time refers to the time during which the business data is cooled in the transaction node.

2. A data transmission device, characterized in that, Applied to management nodes, including: The receiving unit is configured to, upon receiving a business data update request from a preset transaction node, obtain a business data identifier contained in the business data update request; and, based on the business data identifier, invoke a preset external system to obtain the business data corresponding to the business data update request. The execution unit includes: a determination subunit, an allocation subunit, an execution subunit, and a storage subunit; The determining subunit is used to determine the data volume of the business data; The allocation subunit is used to allocate a contiguous memory address space for the service data based on the data volume using a preset address space manager; the address space manager is used to group, divide, and mark a whole address space, and allocate address space according to the data volume of the service data; the capacity of the memory address space allocated for the service data is greater than or equal to the data volume of the service data. The execution subunit is used to request memory from the operating system of the management node that matches the capacity of the memory address space based on the address information of the memory address space, wherein the starting address of the memory is consistent with the address information of the memory address space; The storage subunit is used to store the business data into the allocated memory. The writing unit is used to write the service data already stored in the memory of the management node and the additional information of the service data into the cache area of ​​the management node to obtain cached data to be transmitted. The additional information includes at least the address information of the memory address space. A transmission unit is configured to send the cached data to the transaction node, enabling the transaction node to store the business data in its memory according to the address information of the memory address space, and to write the business data already stored in the transaction node's memory into the transaction node's cache area, so that the transaction process in the transaction node can access the business data; the management node and the transaction node are in the same multicast network; the cached data is transmitted through the multicast network; A cooling unit is used to cool the cache area of ​​the management node after the preset business data expiration time is reached, so as to avoid new access to the cache area; The release unit is used to release the business data and the additional information in the cache area of ​​the management node and reclaim the memory address space after a preset first invalidation cache cooldown time has been reached; wherein, the first invalidation cache cooldown time is greater than the second invalidation cache cooldown time of the business data in the transaction node, the second invalidation cache cooldown time is greater than the maximum allowed transaction time of the transaction process in the transaction node, and the second invalidation cache cooldown time refers to the time during which the business data is cooled in the transaction node.

3. A storage medium, characterized in that, The storage medium includes storage instructions, wherein when the instructions are executed, the device containing the storage medium is controlled to perform the data transmission method as described in claim 1.

4. An electronic device, characterized in that, It includes a memory, and one or more instructions, wherein one or more instructions are stored in the memory and configured to be executed by one or more processors as described in claim 1.

Images