Data transmission method and related equipment

Abstract

The invention discloses a data transmission method and related equipment, which are applied to the field of data processing, and the method comprises the following steps: obtaining a first computing power resource currently used by a first storage system and a to-be-executed task of the first storage system; determining a second computing power resource required by task execution, and determining a first residual computing power resource of the first storage system according to a set computing power resource of the first storage system, the first computing power resource and the second computing power resource; in the first storage system, determining to-be-transmitted first data of the maximum data volume of the first residual computing power resource, and segmenting the first data into a plurality of first sub-data; and transmitting the first sub-data to a second storage system in parallel based on the first residual computing power resource. According to the invention, the data transmission efficiency is improved.

Description

Technical Field

[0001] The present application belongs to the field of data processing technology, and in particular, relates to a data transmission method and related equipment. Background Art

[0002] A large amount of data is stored in the database, which can be used for data query. When a new database is created or a new query service is added to a database, the data in the database with the query service needs to be migrated to the new database or the database with the added query service.

[0003] In the exemplary technology, when data is migrated between databases, one database transfers all the data to another database. However, the amount of data in the database is large, and the time required to transfer the data is long, resulting in low data transmission efficiency. Summary of the invention

[0004] The embodiments of the present application provide a data transmission method and related devices for solving the problem of low data transmission.

[0005] In a first aspect, an embodiment of the present application provides a data transmission method, which is applied to a first storage system. The data transmission method includes:

[0006] Acquire a first computing resource currently used by the first storage system and a task to be executed by the first storage system;

[0007] Determine a second computing resource required for executing the task, and determine a first remaining computing resource of the first storage system according to the set computing resource of the first storage system, the first computing resource, and the second computing resource;

[0008] In the first storage system, determining first data of a maximum data volume to be transmitted by the first remaining computing power resource, and dividing the first data into a plurality of first sub-data;

[0009] Based on the first remaining computing power resources, each of the first sub-data is transmitted in parallel to the second storage system.

[0010] In one embodiment, the step of dividing the first data into a plurality of sub-data includes:

[0011] Determining a second remaining computing power resource of the second storage system;

[0012] Determine an upper limit amount of data that the second storage system can receive using the second remaining computing power resources;

[0013] In response to the maximum data amount being smaller than the upper limit data amount, the first data is divided into a plurality of first sub-data.

[0014] In another embodiment, after determining that the second storage system uses the second remaining computing power resources to receive an upper limit amount of data, the method further includes:

[0015] In response to the maximum data amount being greater than or equal to the upper limit data amount, acquiring second data corresponding to the upper limit data amount in the first storage system, and dividing the second data into a plurality of second sub-data;

[0016] Based on the first remaining computing power resources, each of the second sub-data is transmitted in parallel to the second storage system.

[0017] In another embodiment, the step of dividing the first data into a plurality of first sub-data includes:

[0018] Obtaining a target number of processes in an idle state in the first storage system;

[0019] The first data is divided into the target number of first sub-data.

[0020] In another embodiment, the step of determining the first remaining computing resources of the first storage system according to the set computing resources of the first storage system, the first computing resources, and the second computing resources includes:

[0021] Predicting a target time point for the first storage system to execute the task;

[0022] Determine the interval between the target time point and the current time point, and determine the target time required for the first storage system to transfer all data to the second storage system;

[0023] In response to the interval duration being greater than or equal to the target duration, first remaining computing resources of the first storage system are determined according to the set computing resources of the first storage system, the first computing resources, and the second computing resources.

[0024] In another embodiment, after the step of determining the target time required for the first storage system to transfer all data to the second storage system, the method further includes:

[0025] In response to the interval duration being less than the target duration, determining a difference between the set computing power resource and the first computing power resource;

[0026] The difference is determined as the first remaining computing power resource of the first storage system.

[0027] In a third aspect, an embodiment of the present application provides a first storage system, including:

[0028] An acquisition module, used to acquire a first computing resource currently used by the first storage system and a task to be executed by the first storage system;

[0029] A first determination module, configured to determine a second computing resource required for executing the task, and determine a first remaining computing resource of the first storage system according to the set computing resource of the first storage system, the first computing resource, and the second computing resource;

[0030] A second determination module is used to determine, in the first storage system, first data of a maximum data volume to be transmitted by the first remaining computing power resource, and to divide the first data into a plurality of first sub-data;

[0031] A transmission module is used to transmit each of the first sub-data in parallel to the second storage system based on the first remaining computing power resources.

[0032] In a third aspect, an embodiment of the present application provides a data transmission device, comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the data transmission method as described above is implemented.

[0033] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, characterized in that computer program instructions are stored on the computer-readable storage medium, and when the computer program instructions are executed by a processor, the data transmission method as described above is implemented.

[0034] In a fifth aspect, an embodiment of the present application provides a computer program product. When instructions in the computer program product are executed by a processor of an electronic device, the electronic device is enabled to perform the data transmission method as described above.

[0035] The data transmission method and related equipment of the embodiment of the present application determine the remaining computing resources of the first storage system based on the first computing resources currently used by the first storage system and the second computing resources required for the task to be executed by the first storage system, determine the data of the maximum amount of data to be transmitted by the remaining computing resources in the first storage system, divide the data into multiple sub-data, and then use the remaining computing resources to transmit each sub-data in parallel to the second storage system. In the present application, by dividing the data in the first storage system into multiple sub-data and then transmitting them in parallel to the second storage system, the time for the first storage system to transmit data is reduced and the data transmission efficiency is improved; further, the computing resources for data transmission are the remaining computing resources that can be called by the first storage system, and do not need to occupy the computing resources currently used by the first storage system, so as to avoid the first storage system affecting the services in the first storage system due to data transmission, that is, while ensuring that the services of the first storage system are not affected, the data transmission efficiency is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0037] Figure 1 It is a schematic diagram of the scenario involved in the data transmission method of this application;

[0038] Figure 2 It is a flowchart of the first embodiment of the data transmission method of the present application;

[0039] Figure 3 It is a flowchart of the second embodiment of the data transmission method of the present application;

[0040] Figure 4 It is a flowchart of the third embodiment of the data transmission method of the present application;

[0041] Figure 5 is a flowchart of a fourth embodiment of the data transmission method of the present application;

[0042] Figure 6 It is a functional module diagram of the first storage system of the present application;

[0043] Figure 7 It is a structural schematic diagram of the data transmission device of the present application. DETAILED DESCRIPTION

[0044] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0045] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0046] It should be noted that the acquisition, storage, use and processing of data in the embodiments of the present application are in compliance with the relevant provisions of national laws and regulations.

[0047] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0048] A large amount of data is stored in the database, which can be used for data query. When a new database is created or a query service is added to a database, the data in the database with the query service needs to be migrated to the new database or the database with the query service.

[0049] In the exemplary technology, when data migration is performed between databases, one database transfers all the data to another database.

[0050] The inventors of the present application have discovered that the amount of data in the database is large and the time required to transmit the data is long, resulting in low data transmission efficiency.

[0051] The inventors of the present application therefore conceived that by dividing the data in the first storage system into multiple sub-data and then transmitting them in parallel to the second storage system, the time taken by the first storage system to transmit data is reduced and the data transmission efficiency is improved; further, the computing power resources for data transmission are the remaining computing power resources that can be called upon by the first storage system, and do not need to occupy the computing power resources currently used by the first storage system, thereby avoiding that the services in the first storage system are affected by data transmission. That is, the data transmission efficiency is improved while ensuring that the services of the first storage system are not affected.

[0052] Reference Figure 1 , Figure 1This is a schematic diagram of the scenario of the data transmission method of this application. Figure 1 , the first storage system 100 obtains the first computing power resources currently used by itself and the tasks to be executed. The first storage system 100 determines the second computing power resources required for task execution, and determines the remaining computing power resources of the first storage system 100 by setting the computing power resources, the first computing power resources, and the second computing power resources. Determine the data with the maximum amount of data to be transmitted by the first remaining computing power resources. The first storage system 100 divides the data into multiple sub-data, and uses the first remaining computing power resources to transmit each first sub-data in parallel to the second storage system 200.

[0053] The following combination Figure 1 , the data transmission method of the present application is described in detail.

[0054] Reference Figure 2 , Figure 2 FIG. 1 is a flow chart of a first embodiment of a data transmission method of the present application. The data transmission method includes:

[0055] Step S201: Obtain a first computing resource currently used by a first storage system and a task to be executed by the first storage system.

[0056] In this embodiment, the execution subject is the first storage system. For the convenience of description, the system is used to refer to the first storage system. The system can be a database or a data warehouse.

[0057] When the system needs to migrate its own data to the second storage system, the system obtains the computing resources currently used by itself, which are defined as the first computing resources. Computing resources refer to the ability of the system to process data, and computing resources can be measured by numerical values.

[0058] The system determines the task currently being executed. The resources occupied by each task are the computing power resources, and the sum of the computing power resources occupied by each task is the first computing power resource.

[0059] After obtaining the first computing power resource, the system determines the current task to be executed, which may be one or more tasks.

[0060] Step S202, determine the second computing resources required for task execution, and determine the first remaining computing resources of the first storage system according to the set computing resources, the first computing resources and the second computing resources of the first storage system.

[0061] After determining the task to be executed, the system determines the second computing power resources required for the task. Exemplarily, the system stores a task execution log, and the system extracts the previous execution information of the task from the task execution log, and obtains the computing power resources required for the task to be executed from the execution information as the second computing power resources. When the task execution log does not include the previous execution information of the task to be executed, the task description content of the task to be executed is obtained, and the computing power resources required for the execution of the task are converted based on the task description content as the second computing power resources. For example, the task description task can be input into the prediction model to obtain the second computing power resources output by the prediction model. The prediction model is trained based on the task description content of the completed task and the computing power resources of the completed task.

[0062] A set computing power resource is provided in the system, and the set computing power resource may be the maximum computing power resource in the system, or the set computing power resource may be determined based on the maximum computing power resource. For example, the set computing power resource may be obtained by subtracting a value from the maximum computing power resource. The system determines the remaining computing power resources of the system based on the set computing power resources, the first computing power resources, and the second computing power resources, and the remaining computing power resources are defined as the first remaining computing power resources. In one example, the first remaining computing power resources are subtracted from the set computing power resources, and the second computing power resources are subtracted to obtain the first remaining computing power resources. In another example, the sum of the first computing power resources and the second computing power resources is calculated as the busy computing power resources, and the system then determines the difference between the set computing power resources and the busy computing power resources, and the computing power resources less than the difference are used as the first remaining computing power resources. As can be seen from the above, the first remaining computing power resources refer to the maximum computing power resources that the system can use for data transmission.

[0063] Step S203: In the first storage system, determine the first data with the maximum data volume to be transmitted by the first remaining computing power resources, and divide the first data into a plurality of first sub-data.

[0064] After determining the first remaining computing power resource, the system determines the maximum amount of data that the first remaining computing power resource can process. Exemplarily, the system stores the amount of data S corresponding to the data that can be transmitted by the unit computing power resource L, and the maximum amount of data = (first remaining computing power resource / L)*S. The system divides the data corresponding to the maximum amount of data from the data stored in itself as the first data.

[0065] After obtaining the first data, the system divides the first data into multiple sub-data, each of which is defined as first sub-data. The system can call a third-party service to divide the first data into multiple first sub-data.

[0066] Step S204: Based on the first remaining computing power resources, each first sub-data is transmitted in parallel to the second storage system.

[0067] After obtaining each first sub-data, the system uses the first remaining computing power resources to transmit each first sub-data in parallel to the second storage system. The second storage system refers to the database that receives the system migration data. The second storage system can be a newly created database; or the second storage system is a database for newly added services, and the newly added services are the services currently owned by the system. Therefore, the system needs to migrate its own data to the second storage system as the historical data of the second storage system, so that the second storage system can open new services to users based on the migrated data.

[0068] It should be noted that when not all data in the system has been migrated, the system uses the first remaining computing resources to transfer each first sub-data to the second storage system, and then the system returns to execute steps S201 to S204 until all data to be migrated in the system has completed the migration.

[0069] In this embodiment, based on the first computing power resources currently used by the first storage system and the second computing power resources required for the tasks to be performed by the first storage system, the remaining computing power resources of the first storage system are determined, and the data with the maximum amount of data to be transmitted by the remaining computing power resources in the first storage system is determined, and the data is divided into multiple sub-data, so that the remaining computing power resources are used to transmit each sub-data in parallel to the second storage system. In this embodiment, by dividing the data in the first storage system into multiple sub-data and then transmitting them in parallel to the second storage system, the time for the first storage system to transmit data is reduced, and the data transmission efficiency is improved; further, the computing power resources for data transmission are the remaining computing power resources that can be called by the first storage system, and do not need to occupy the computing power resources currently used by the first storage system, so as to avoid the first storage system affecting the services in the first storage system due to data transmission, that is, while ensuring that the services of the first storage system are not affected, the data transmission efficiency is improved.

[0070] Reference Figure 3 , Figure 3 This is a flow chart of the second embodiment of the data transmission method of the present application. Based on the first embodiment, step S203 includes:

[0071] Step S301, determine the current second remaining computing power resources of the second storage system.

[0072] In this embodiment, the second storage system also has services, and the system transfers data to the second storage system, and it is also necessary to ensure that the services of the second storage system will not be affected.

[0073] In response to this, the system sends a data acquisition request to the second storage system. The data acquisition request user requests the second storage system to feedback the value of the remaining computing power resources to the system. That is, the system receives the relevant information of the computing power resources fed back by the second storage system. Based on the relevant information, the remaining computing power resources of the second storage system can be determined. The remaining computing power resources of the second storage system are defined as the second remaining computing power resources.

[0074] Step S302, determining the upper limit of the amount of data received by the second storage system using the second remaining computing power resources.

[0075] In this embodiment, the second storage system uses the second remaining computing power resources to receive data. In order not to affect the service of the second storage system by the received data, the system needs to determine the upper limit of the amount of data that the second storage system can receive using the second remaining computing power resources. Exemplarily, the system stores a mapping relationship between computing power resources and the amount of data received by the computing power resources. Based on the mapping relationship and the second remaining computing power resources, the maximum amount of data that the second storage system can receive using the second remaining computing power resources can be determined. The maximum amount of data is defined as the upper limit amount of data.

[0076] Step S303 , in response to the maximum data amount being smaller than the upper limit data amount, dividing the first data into a plurality of first sub-data.

[0077] The system compares the maximum data volume with the upper limit data volume. When the maximum data volume is less than the upper limit data volume, it can be determined that the second storage system receiving the first data will not affect the service of the second storage system. The system then divides the first data into multiple first sub-data.

[0078] When the maximum data volume is greater than or equal to the upper limit data volume, it can be determined that the data volume of the first data received by the second storage system is large, or occupies the computing power resources currently used by the second storage system, and the system obtains the data corresponding to the upper limit data volume from the stored data as the second data, ensuring that the service of the second storage system will not be affected by receiving the data. After obtaining the second data, the system divides the second data into multiple second sub-data, and then uses the first remaining computing power resources to transmit each second sub-data in parallel to the second storage system.

[0079] In this embodiment, the upper limit data amount is compared with the maximum data amount based on the second remaining computing resources and the first remaining computing resources of the second storage system. While ensuring that the services of the first storage system and the second storage system are not affected in advance, the data in the first storage system is quickly transmitted to the second storage system, thereby improving the data transmission efficiency.

[0080] Reference Figure 4 , Figure 4This is a flowchart of a third embodiment of the data transmission method of the present application. Based on the first or second embodiment, step S203 includes:

[0081] Step S401: Obtain a target number of processes in an idle state in a first storage system.

[0082] Step S402: dividing the first data into a target number of first sub-data.

[0083] In this embodiment, the system sends sub-data to the second storage system in parallel, and the system can use a process to transmit a sub-data, so the system can split the first data based on the number of processes. Exemplarily, the system determines the number of processes in its own idle state, and the data volume is defined as the target number.

[0084] After obtaining the target number, the system divides the first data into the target number of first sub-data. During data transmission, based on the first remaining computing power resources and the target number of idle processes, each first sub-data is transmitted to the second storage system, that is, each idle process transmits a sub-data.

[0085] It should be noted that idle processes include processes that have not been assigned tasks, and processes that have been assigned tasks and meet the conditions. Specifically, the system determines each first process that has been assigned tasks but not executed, and predicts the start time point of the first process to execute the task, and then calculates the interval between the current time point and the start time point. When the interval is longer than the maximum time required for the system to transmit data, the first process is determined as an idle process. The predicted start time point of the first process to execute the task can be confirmed by the task description information of the task. For example, the task description information specifies that the task needs to be completed at X time. The system obtains the previous execution records of the task. The historical execution time of the task can be determined through the execution records. The start time point can be obtained by pushing X time forward by the historical execution time.

[0086] In this embodiment, the system obtains the target number of idle processes in the first storage system, and then divides the first data into the target number of first sub-data, so that the idle processes can transfer the corresponding first sub-data to the second storage system.

[0087] Reference Figure 5 , Figure 5 This is a flowchart of a fourth embodiment of the data transmission method of the present application, based on any one of the first to third embodiments, step S202 includes:

[0088] Step S501: predicting a target time point for a first storage system to execute a task.

[0089] In this embodiment, the system predicts the target time point for the first storage system to execute the task. The prediction of the target time point is the same as the prediction of the start time point for the first process to execute the task, which will not be described in detail here.

[0090] Step S502: determine the interval between the target time point and the current time point, and determine the target time required for the first storage system to transfer all data to the second storage system.

[0091] After obtaining the target time point of the task, the system determines the interval between the current time point and the target time point, and determines the time required for the system to transfer all data to the second storage system, which is defined as the target time. Exemplarily, the system first determines the total amount of all its own data, calculates the amount of data sent in parallel each time, and determines the ratio between the total amount of data and the amount of data, and the time for transmitting one data is D, and the product of the ratio and D is the target time.

[0092] Step S503, in response to the interval duration being greater than or equal to the target duration, determining the first remaining computing resources of the first storage system according to the set computing resources, the first computing resources, and the second computing resources of the first storage system.

[0093] The system determines whether the interval duration is less than the target duration. When the interval duration is greater than or equal to the target duration, the second computing power resource required for task execution cannot be freed up for data transmission, so the computing power resource is set to subtract the first computing power resource and the second computing power resource to obtain the first remaining computing power resource.

[0094] When the interval duration is less than the target duration, the system can first borrow the second computing power resource for data transmission. Therefore, the system determines the difference between the set computing power resource and the first computing power resource, and determines the difference as the first remaining computing power resource for data transmission.

[0095] In this embodiment, the system predicts the target time point for executing the task, and thereby determines, through the interval between the target time point and the current time point, that the second computing power resources required to execute the task can be used to transmit data, thereby ensuring that the system uses the most computing power resources to transmit data and improving data transmission efficiency.

[0096] See also Figure 6 , the first storage system 600 provided in the embodiment of the present application, the first storage system 600 includes:

[0097] An acquisition module 610 is used to acquire a first computing resource currently used by the first storage system and a task to be executed by the first storage system;

[0098] A first determination module 620, configured to determine the second computing resources required for task execution, and determine the first remaining computing resources of the first storage system according to the set computing resources, the first computing resources, and the second computing resources of the first storage system;

[0099] A second determination module 630 is used to determine, in the first storage system, first data of the maximum data volume to be transmitted by the first remaining computing power resource, and divide the first data into a plurality of first sub-data;

[0100] The transmission module 640 is used to transmit each first sub-data in parallel to the second storage system based on the first remaining computing power resource.

[0101] In one embodiment, the first storage system 600 is specifically used for:

[0102] Determine the current second remaining computing power resource of the second storage system;

[0103] Determine an upper limit amount of data received by the second storage system using the second remaining computing power resources;

[0104] In response to the maximum data amount being less than the upper limit data amount, the first data is divided into a plurality of first sub-data.

[0105] In one embodiment, the first storage system 600 is specifically used for:

[0106] In response to the maximum data amount being greater than or equal to the upper limit data amount, obtaining second data corresponding to the upper limit data amount in the first storage system, and dividing the second data into a plurality of second sub-data;

[0107] Based on the first remaining computing power resources, each second sub-data is transmitted in parallel to the second storage system.

[0108] In one embodiment, the first storage system 600 is specifically used for:

[0109] Obtaining a target number of processes in an idle state in the first storage system;

[0110] The first data is divided into a target number of first sub-data.

[0111] In one embodiment, the first storage system 600 is specifically used for:

[0112] Predicting a target time point for the first storage system to execute a task;

[0113] Determine the interval between the target time point and the current time point, and determine the target time required for the first storage system to transfer all data to the second storage system;

[0114] In response to the interval duration being greater than or equal to the target duration, a first remaining computing resource of the first storage system is determined according to the set computing resource, the first computing resource, and the second computing resource of the first storage system.

[0115] In one embodiment, the first storage system 600 is specifically used for:

[0116] In response to the interval duration being less than the target duration, determining a difference between the set computing power resource and the first computing power resource;

[0117] The difference is determined as the first remaining computing power resource of the first storage system.

[0118] Each module of the first storage system provided in the embodiment of the present application can realize Figures 2 to 5 Any embodiment provides the functions of each step of the data transmission method and can achieve its corresponding technical effects. For the sake of concise description, they will not be repeated here.

[0119] Figure 7 A schematic diagram of the hardware structure of a data transmission device provided in an embodiment of the present application is shown.

[0120] The data transmission device may include a processor 701 and a memory 702 storing computer program instructions.

[0121] Specifically, the processor 701 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0122] The memory 702 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 702 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 702 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 702 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 702 is a non-volatile solid-state memory.

[0123] In certain embodiments, the memory 702 may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage media device, an optical storage media device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Thus, in general, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0124] The processor 701 implements any one of the authentication methods in the above embodiments by reading and executing computer program instructions stored in the memory 702 .

[0125] In one example, the data transmission device may further include a communication interface 703 and a bus 710. Figure 7 As shown, the processor 701, the memory 702, and the communication interface 703 are connected via a bus 710 and communicate with each other.

[0126] The communication interface 703 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0127] Bus 710 includes hardware, software or both, and the parts of online data flow billing equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front-end bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 710 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnection.

[0128] The data transmission method in the above embodiment can be implemented by providing a computer storage medium in the present application embodiment. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the data transmission methods in the above embodiment is implemented.

[0129] An embodiment of the present application further provides a computer program product, including a computer program, which implements any one of the data transmission methods in the above embodiments when the computer program is processed and executed.

[0130] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0131] Aspects of the present disclosure are described above with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data transmission device to produce a machine so that these instructions executed by the processor of the computer or other programmable data transmission device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0132] The above are only specific implementation methods of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A data transmission method, characterized in that: Applied to the first storage system, the data transmission method includes: Acquire a first computing resource currently used by the first storage system and a task to be executed by the first storage system; Determine a second computing resource required for executing the task, and determine a first remaining computing resource of the first storage system according to the set computing resource of the first storage system, the first computing resource, and the second computing resource; In the first storage system, determining first data of a maximum data volume to be transmitted by the first remaining computing power resource, and dividing the first data into a plurality of first sub-data; Based on the first remaining computing power resources, each of the first sub-data is transmitted in parallel to the second storage system.

2. The data transmission method according to claim 1, characterized in that: The step of dividing the first data into a plurality of sub-data comprises: Determining a second remaining computing power resource of the second storage system; Determine an upper limit amount of data that the second storage system can receive using the second remaining computing power resources; In response to the maximum data amount being smaller than the upper limit data amount, the first data is divided into a plurality of first sub-data.

3. The data transmission method according to claim 2, characterized in that: After determining that the second storage system uses the second remaining computing power resources to receive an upper limit amount of data, the method further includes: In response to the maximum data amount being greater than or equal to the upper limit data amount, acquiring second data corresponding to the upper limit data amount in the first storage system, and dividing the second data into a plurality of second sub-data; Based on the first remaining computing power resources, each of the second sub-data is transmitted in parallel to the second storage system.

4. The data transmission method according to claim 1, characterized in that: The step of dividing the first data into a plurality of first sub-data comprises: Obtaining a target number of processes in an idle state in the first storage system; The first data is divided into the target number of first sub-data.

5. The data transmission method according to claim 1, characterized in that: The step of determining the first remaining computing resources of the first storage system according to the set computing resources of the first storage system, the first computing resources, and the second computing resources includes: Predicting a target time point for the first storage system to execute the task; Determine the interval between the target time point and the current time point, and determine the target time required for the first storage system to transfer all data to the second storage system; In response to the interval duration being greater than or equal to the target duration, first remaining computing resources of the first storage system are determined according to the set computing resources of the first storage system, the first computing resources, and the second computing resources.

6. The data transmission method according to claim 5, characterized in that: After the step of determining the target time required for the first storage system to transfer all data to the second storage system, the method further includes: In response to the interval duration being less than the target duration, determining a difference between the set computing power resource and the first computing power resource; The difference is determined as the first remaining computing power resource of the first storage system.

7. A first storage system, characterized in that: include: An acquisition module, used to acquire a first computing resource currently used by the first storage system and a task to be executed by the first storage system; A first determination module, configured to determine a second computing resource required for executing the task, and determine a first remaining computing resource of the first storage system according to the set computing resource of the first storage system, the first computing resource, and the second computing resource; A second determination module is used to determine, in the first storage system, first data of a maximum data volume to be transmitted by the first remaining computing power resource, and to divide the first data into a plurality of first sub-data; A transmission module is used to transmit each of the first sub-data in parallel to the second storage system based on the first remaining computing power resources.

8. A data transmission device, characterized in that: The device comprises: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the data transmission method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the data transmission method according to any one of claims 1 to 6 is implemented.

10. A computer program product, characterized in that When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device is enabled to execute the data transmission method as described in any one of claims 1 to 6.

Images