Data uploading method and device and electronic equipment

Abstract

The invention relates to the technical field of cloud computing, in particular to a data uploading method and device and electronic equipment. The method comprises the following steps: when a file uploading request which is sent by a client and is used for uploading a preset file is received, acquiring at least two uploading nodes and actual performance data of each uploading node; determining a node weight corresponding to each uploading node based on the actual performance data; taking the uploading nodes of which the node weights are greater than a preset weight as candidate nodes capable of processing the file uploading request; and receiving the preset file through one or more nodes in the candidate nodes.

Description

Technical Field

[0001] The present disclosure relates to the technical field of cloud computing, and in particular to a data uploading method, device and electronic device. Background Art

[0002] In modern Internet applications, file upload service is a very critical function. Whether it is a cloud storage service, file sharing platform, content management system or social media platform, it needs to provide users with an efficient and reliable file upload experience.

[0003] Currently, most file upload services allocate upload nodes based on static configuration or load balancing strategies (such as polling or random selection). These methods are relatively simple, but cannot dynamically adapt to the performance changes of actual upload nodes. For example, a node may be overloaded during peak hours, resulting in a drop in upload speed or an increase in error rate, while other nodes may be idle and not fully utilized.

[0004] Therefore, how to improve the node utilization of upload nodes has become an urgent problem to be solved. Summary of the invention

[0005] In order to solve the above technical problems, the present disclosure provides a data uploading method, device and electronic device.

[0006] The technical solution of the present disclosure is as follows:

[0007] In a first aspect, the present disclosure provides a data upload method, comprising: upon receiving a file upload request sent by a client for uploading a preset file, obtaining at least two upload nodes and actual performance data of each upload node; determining a node weight corresponding to each upload node based on the actual performance data; wherein the node weight is inversely proportional to the node utilization corresponding to the actual performance data; taking an upload node whose node weight is greater than a preset weight as a candidate node that can process the file upload request; and receiving the preset file through one or more of the candidate nodes.

[0008] In a second aspect, the present disclosure provides a data uploading device, including: a processing unit, which is used to control the transceiver unit to obtain at least two upload nodes and actual performance data of each upload node when receiving a file upload request sent by a client for uploading a preset file; the processing unit is also used to determine the node weight corresponding to each upload node based on the actual performance data obtained by the transceiver unit; wherein the node weight is inversely proportional to the node utilization rate corresponding to the actual performance data; the processing unit is also used to use the upload node whose node weight is greater than the preset weight as a candidate node that can process the file upload request; the processing unit is also used to receive the preset file through one or more nodes of the candidate node

[0009] In a third aspect, the present disclosure provides an electronic device, comprising: a memory and a processor, the memory being used to store a computer program; the processor being used to enable the electronic device to implement any one of the data uploading methods provided in the second aspect when executing the computer program.

[0010] In a fourth aspect, the present invention provides a computer-readable storage medium, comprising: a computer program is stored on the computer-readable storage medium, and the computer program is executed by a processor as any one of the data uploading methods provided in the second aspect.

[0011] In a fifth aspect, the present invention provides a computer program product. When the computer program product is run on a computer, the computer executes any one of the data uploading methods provided in the second aspect.

[0012] It should be noted that the above computer instructions may be stored in whole or in part on a first computer-readable storage medium, wherein the first computer-readable storage medium may be packaged together with the processor of the electronic device, or may be packaged separately from the processor of the electronic device, which is not limited in the present disclosure.

[0013] The description of the second, third, fourth and fifth aspects in the present disclosure can refer to the detailed description of the first aspect; and the beneficial effects of the description of the second, third, fourth and fifth aspects can refer to the beneficial effect analysis of the first aspect, which will not be repeated here.

[0014] In the present disclosure, the names of the above electronic devices do not limit the devices or functional modules themselves. In actual implementation, these devices or functional modules may appear with other names. As long as the functions of each device or functional module are similar to those of the present disclosure, they fall within the scope of the claims of the present disclosure and their equivalent technologies.

[0015] These and other aspects of the present disclosure will become more apparent from the following description.

[0016] Compared with the prior art, the technical solution provided by the present invention has the following advantages:

[0017] The data upload method provided by the present disclosure, upon receiving a file upload request sent by a client for uploading a preset file, obtains at least one upload node and the actual performance data of each upload node; then, based on the actual performance data, determines the node weight corresponding to each upload node. Since the node weight is inversely proportional to the node utilization corresponding to the actual performance data; in this way, the upload nodes can be screened, such as taking the upload nodes whose node weight is greater than the preset weight as candidate nodes; thereby, the upload nodes with lower node utilization can be used to process the file upload request, thereby improving the node utilization of the upload node. Afterwards, an indication information containing the candidate node is sent to the preset user account; wherein the indication information is used to instruct the client to upload the preset file through one or more upload nodes corresponding to the candidate node. The node weight can be used to screen out the upload nodes with lower node utilization, thereby increasing the number of file upload requests processed by the upload node, thereby improving the node utilization of the upload node. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0020] Figure 1 One of the flowcharts of the data uploading method provided in the embodiment of the present disclosure;

[0021] Figure 2 The second flowchart of the data uploading method provided in the embodiment of the present disclosure;

[0022] Figure 3 The third flowchart of the data uploading method provided in the embodiment of the present disclosure;

[0023] Figure 4 A fourth flowchart of the data uploading method provided in an embodiment of the present disclosure;

[0024] Figure 5 A fifth flowchart of the data uploading method provided in the embodiment of the present disclosure;

[0025] Figure 6 A schematic diagram of the structure of a data uploading device provided in an embodiment of the present disclosure;

[0026] Figure 7A schematic diagram of the structure of a server provided in an embodiment of the present disclosure;

[0027] Figure 8 A schematic diagram of the structure of a computer program product of a data uploading method provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0029] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0030] The display device provided in the embodiments of the present application may have various implementation forms, for example, it may be a television, a smart TV, a laser projection device, a monitor, an electronic whiteboard, an electronic table, etc. Figure 1 and Figure 2 This is a specific implementation of the display device of the present application.

[0031] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0032] Exemplarily, the data upload method provided by the embodiment of the present disclosure is described by taking the execution subject of the data upload method provided by the embodiment of the present disclosure as a server as an example.

[0033] Figure 1 is a flow chart of a data uploading method according to an exemplary embodiment. Figure 1 As shown, including: S11-S14.

[0034] S11. Upon receiving a file upload request for uploading a preset file sent by a client, obtaining at least one upload node and actual performance data of each upload node.

[0035] In some examples, a user may install a client for uploading preset files on an electronic device, and after the client is started, a communication connection is established with a server. In this way, the user can upload the preset file through the client. Later, when the user needs to upload the preset file, the user can perform a long transfer operation of the preset file on the client. At this time, the client responds to the user's upload operation of the preset file and sends a file upload request for uploading the preset file to the server. Later, when the server receives the file upload request for uploading the preset file sent by the client, it obtains at least one upload node that can process the file upload request, and the actual performance data of each upload node.

[0036] In some examples, obtaining at least one upload node that can process a file upload request includes:

[0037] At least one upload node that can process the file upload request can be determined based on an Internet Protocol Address (IP) address in the file upload request. For example, taking the first coordinate (such as longitude and latitude) corresponding to the IP address as the center, the distance between the second coordinate (such as longitude and latitude) corresponding to the upload node and the first coordinate is calculated. The upload nodes whose distance is less than or equal to the preset distance are screened, and the upload nodes whose distance is less than or equal to the preset distance are used as at least one upload node that can process the file upload request.

[0038] Alternatively, a correspondence table between preset coordinate intervals and node intervals is preconfigured in the memory of the server, and the node interval contains at least one upload node. The server can query the correspondence table based on the first coordinate, determine the preset coordinate interval that the first coordinate falls into, and use the upload node in the node interval corresponding to the preset coordinate interval as at least one upload node that can process the file upload request.

[0039] S12. Based on the actual performance data, determine the node weight corresponding to each uploading node. The node weight is inversely proportional to the node utilization corresponding to the actual performance data;

[0040] In some examples, the actual performance data includes total upload traffic, average upload rate, upload error rate, average CPU usage, and average memory usage.

[0041] In some examples, the actual performance data of the currently calculated upload node can be input into the weight calculation formula for weight calculation to obtain the node weight corresponding to the currently calculated upload node; wherein the weight calculation formula is obtained based on the historical performance data fitting. The process of obtaining the weight calculation formula is as follows:

[0042] Obtaining historical performance data of at least one upload node capable of uploading files in each historical period; wherein the historical performance data includes total upload traffic, average upload rate, upload error rate, average CPU usage, and average memory usage;

[0043] Based on the ratio of the total upload traffic, average upload rate, upload error rate, average CPU usage and average memory usage in all historical performance data, determine the first coefficient of the total upload traffic, the second coefficient of the average upload rate, the third coefficient of the upload error rate, the fourth coefficient of the average CPU usage and the fifth coefficient of the average memory usage. For example, if the total upload traffic is 1500MB, the average upload rate is 1.5MB / s, the upload error rate is 0.1%, the average CPU usage is 30%, and the average memory usage is 25%, then the ratio of the total upload traffic, the average upload rate, the upload error rate, the average CPU usage and the average memory usage is 1500000:1500:1:300:250, that is, the first coefficient is 1500000, the second coefficient is 1500, the third coefficient is 1, the fourth coefficient is 300, and the fifth coefficient is 250.

[0044] Fitting is performed based on the first coefficient, the second coefficient, the third coefficient, the fourth coefficient and the fifth coefficient to obtain a weight calculation formula.

[0045] Exemplarily, the weight calculation formula may be:

[0046] node w =-α1×traffic+α2×errorRate+α3×speed±α4×cpuUtil+α5×memoryUtil.

[0047] Among them, node w represents the node weight, α1 represents the first coefficient, α2 represents the second coefficient, α3 represents the third coefficient, α4 represents the fourth coefficient, α5 represents the fifth coefficient, trafffc represents the total upload traffic in the actual performance data, errorRate represents the upload error rate in the actual performance data, speed represents the average upload rate in the actual performance data, cpuUtil represents the average CPU usage in the actual performance data, and memoryUtil represents the average memory usage in the actual performance data.

[0048] In this way, the actual performance data of the upload node currently being calculated can be input into the weight calculation formula to perform weight calculation, thereby obtaining the node weight corresponding to the upload node currently being calculated.

[0049] In some examples, the total upload traffic is equal to the sum of the traffic uploaded in the current period.

[0050] In some examples, the average upload rate is equal to the ratio of the total upload traffic to the duration corresponding to the current cycle.

[0051] In some examples, the upload error rate is equal to the ratio of the number of preset files that are not correctly uploaded in the current period to the number of preset files uploaded in the current period.

[0052] In some examples, the average CPU usage is equal to the ratio of the CPU usage obtained at each moment in the current cycle to the total number of moments included in the current cycle.

[0053] In some examples, the average memory usage is equal to the ratio of the memory usage obtained at each moment in the current cycle to the total number of moments included in the current cycle.

[0054] It should be noted that the node weight is inversely proportional to the total amount of file upload requests currently processed by the upload node. That is, the larger the total amount of file upload requests currently processed by the upload node, the smaller the corresponding node weight, and the lower the probability of the upload node being selected as a candidate node, thereby optimizing the node utilization of the upload node.

[0055] In some examples, the actual performance data of the upload node can be input into the weight analysis model for weight analysis to determine the node weight corresponding to the upload node. The training process of the weight analysis model is as follows:

[0056] Acquire training sample data and a marking result of the training sample data; wherein the training sample data includes at least one historical performance data, and the marking result includes a node weight corresponding to the historical performance data.

[0057] The training sample data is input into the neural network model for learning, and the prediction results of the neural network model for the training sample data are obtained.

[0058] Based on the prediction results and labeling results, the network parameters of the neural network model are adjusted until the neural network model converges to obtain a weight analysis model.

[0059] In some examples, the node utilization of the upload node is equal to the number of file upload requests processed within a preset period.

[0060] S13: taking an upload node whose node weight is greater than a preset weight as a candidate node that can process the file upload request.

[0061] S14. Receive a preset file through one or more candidate nodes.

[0062] In some examples, after determining the candidate node, the server needs to send indication information containing the candidate node to the preset user account; wherein the indication information is used to instruct the client to upload the preset file through one or more upload nodes corresponding to the candidate node. When the candidate node contains multiple upload nodes, the client can upload the preset file in parallel through the multiple upload nodes contained in the candidate node, such as: the client divides the preset file into a target number of copies, and the target number of copies is equal to the total number of multiple upload nodes contained in the candidate node. Afterwards, the client uploads a split file after the preset file is divided to each upload node.

[0063] For example, the candidate nodes include three upload nodes, namely upload node 1, upload node 2 and upload node 3, and the target number of copies is determined to be 3. The client needs to split the preset file into three copies, namely split file 1, split file 2 and split file 3. At this time, the client can upload split file 1 to upload node 1, upload split file 2 to upload node 2, and upload split file 3 to upload node 3.

[0064] Afterwards, when the user needs to download the preset file, the download operation of the preset file can be performed on the client. At this time, the client sends a download request for downloading the preset file to the server in response to the download operation of the preset file. After receiving the download request sent by the client, the server determines the upload node (such as upload node 1, upload node 2 and upload node 3) storing the preset file. Afterwards, the server sends the upload node storing the preset file to the client. After receiving the upload node storing the preset file sent by the server, the client downloads split file 1 through upload node 1, downloads split file 2 through upload node 2, and downloads split file 3 through upload node 3, thus completing the download of the preset file.

[0065] As can be seen from the above, the data upload method provided by the embodiment of the present disclosure obtains at least one upload node and the actual performance data of each upload node when receiving a file upload request sent by a client for uploading a preset file; then, based on the actual performance data, the node weight corresponding to each upload node is determined; since the node weight is inversely proportional to the node utilization corresponding to the actual performance data, the higher the node utilization of the upload node, the lower the corresponding node weight, and the lower the node utilization of the upload node, the higher the corresponding node weight. In this way, the upload nodes can be screened, such as taking the upload nodes with node weights greater than the preset weights as candidate nodes; thus, the upload nodes with lower node utilization can be used to process the file upload request, thereby improving the node utilization of the upload node. Afterwards, the indication information containing the candidate node is sent to the preset user account; wherein the indication information is used to instruct the client to upload the preset file through one or more upload nodes corresponding to the candidate node. The upload nodes with lower node utilization can be screened out by the node weight, thereby increasing the number of file upload requests processed by the upload node, thereby improving the node utilization of the upload node.

[0066] In some feasible examples, the file upload request includes an Internet Protocol address; Figure 1 ,like Figure 2 As shown, the above S11 can be specifically implemented through the following S110 and S111.

[0067] S110. When receiving a file upload request sent by a client for uploading a preset file, obtain an Internet Protocol address in the file upload request.

[0068] S111. Based on the Internet Protocol address, obtain actual performance data of each upload node in the service area corresponding to the Internet Protocol address in the current cycle.

[0069] In some possible implementation examples, combined with Figure 1 ,like Figure 3 As shown, the above S12 can be specifically implemented through the following S120 and S121.

[0070] S120: Send the actual performance data to a preset message queue.

[0071] In some examples, in order to reduce the occupancy of computing resources, after the actual performance data is obtained, the actual performance data is sent to a preset message queue, thereby improving the response speed of the system.

[0072] In some examples, the preset message queue can be a Kafka message queue.

[0073] S121. Use a preset stream processing framework to process actual performance data in a preset message queue according to a preset period, and determine a node weight corresponding to each uploading node.

[0074] In some examples, the preset stream processing framework can be a Flink stream processing framework.

[0075] In some examples, the preset period may be 10 minutes.

[0076] In some possible implementation examples, combined with Figure 1 ,like Figure 4 As shown, the above S12 can be specifically implemented through the following S122.

[0077] S122. For each upload node, perform the following operations: input the actual performance data of the upload node into the weight calculation formula to perform weight calculation, and obtain the node weight corresponding to the currently calculated upload node; wherein the weight calculation formula is obtained based on the fitting of historical performance data.

[0078] In some possible implementation examples, combined with Figure 1 ,like Figure 5 As shown, before executing S11, the data uploading method provided by the embodiment of the present disclosure also includes S15-S17.

[0079] S15, obtaining historical performance data of at least one upload node that can process file upload requests in each cycle; wherein the historical performance data includes total upload traffic, average upload rate, upload error rate, average CPU usage, and average memory usage;

[0080] S16, based on the ratio of the total upload traffic, the average upload rate, the upload error rate, the average CPU usage rate and the average memory usage rate in all historical performance data, determine a first coefficient of the total upload traffic, a second coefficient of the average upload rate, a third coefficient of the upload error rate, a fourth coefficient of the average CPU usage rate and a fifth coefficient of the average memory usage rate;

[0081] S17. Fitting is performed based on the first coefficient, the second coefficient, the third coefficient, the fourth coefficient and the fifth coefficient to obtain a weight calculation formula.

[0082] In some feasible examples, the actual performance data includes total upload traffic, average upload rate, upload error rate, average CPU usage, and average memory usage.

[0083] The above mainly introduces the solution provided by the embodiment of the present invention from the perspective of the method. In order to achieve the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present invention can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0084] The embodiment of the present invention can divide the data upload device into functional modules according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present invention is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0085] like Figure 6 As shown, an embodiment of the present invention provides a structural diagram of a data upload device 10. The data upload device 10 includes: a processing unit 101 and a transceiver unit 102.

[0086] The processing unit 101 is used to control the transceiver unit 102 to obtain at least one upload node and actual performance data of each upload node when receiving a file upload request sent by a client to upload a preset file; the processing unit 101 is also used to determine the node weight corresponding to each upload node based on the actual performance data obtained by the transceiver unit 102; wherein the node weight is inversely proportional to the node utilization corresponding to the actual performance data; the processing unit 101 is also used to use the upload node whose node weight is greater than the preset weight as a candidate node that can process the file upload request; the processing unit 101 is also used to receive the preset file through one or more nodes among the candidate nodes.

[0087] In some feasible examples, the file upload request includes an Internet Protocol address; the transceiver unit 102 is specifically used to obtain the Internet Protocol address in the file upload request; the processing unit 101 is specifically used to obtain, based on the Internet Protocol address obtained by the transceiver unit 102, actual performance data of each uploading node in the service area corresponding to the Internet Protocol address in the current cycle.

[0088] In some feasible examples, the processing unit 101 is specifically used to send the actual performance data to a preset message queue; the processing unit 101 is specifically used to use a preset stream processing framework to process the actual performance data in the preset message queue according to a preset period, and determine the node weight corresponding to each uploading node.

[0089] In some implementable examples, the processing unit 101 is specifically used to perform the following operations for each upload node: input the actual performance data of the upload node into the weight calculation formula for weight calculation, and obtain the node weight corresponding to the currently calculated upload node; wherein the weight calculation formula is obtained based on the fitting of historical performance data.

[0090] In some implementable examples, the transceiver unit 102 is also used to obtain historical performance data of at least one upload node that can process file upload requests in each cycle; wherein the historical performance data includes total upload traffic, average upload rate, upload error rate, average CPU usage rate and average memory usage rate; the processing unit 101 is also used to determine a first coefficient of total upload traffic, a second coefficient of average upload rate, a third coefficient of upload error rate, a fourth coefficient of average CPU usage rate and a fifth coefficient of average memory usage rate based on the ratio of total upload traffic, average upload rate, upload error rate, average CPU usage rate and average memory usage rate in all historical performance data; the processing unit 101 is also used to perform fitting based on the first coefficient, the second coefficient, the third coefficient, the fourth coefficient and the fifth coefficient to obtain a weight calculation formula.

[0091] In some feasible examples, the actual performance data includes total upload traffic, average upload rate, upload error rate, average CPU usage, and average memory usage.

[0092] Of course, the data upload device 10 provided in the embodiment of the present invention includes but is not limited to the above modules, for example, the data upload device 10 may also include a storage unit 103. The storage unit 103 may be used to store the program code of the data upload device 10, and may also be used to store data generated by the data upload device 10 during operation, such as data in a write request.

[0093] Figure 7 A schematic diagram of the structure of a server provided in an embodiment of the present invention, such as Figure 7 As shown, the server may include: at least one processor 51, a memory 52, a communication interface 53, a communication bus 54 and a display screen 55. Figure 7 A detailed introduction to each component of the server:

[0094] The processor 51 is the control center of the server, and may be a processor or a general term for multiple processing elements. For example, the processor 51 is a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more DSPs, or one or more field programmable gate arrays (FPGAs).

[0095] In a specific implementation, as an embodiment, the processor 51 may include one or more CPUs, such as Figure 7 Also, as an embodiment, the server may include multiple processors, such as Figure 7 51 and processor 56 are shown in FIG. Each of these processors may be a single-core processor (Single-CPU) or a multi-core processor (Multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0096] The memory 52 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 52 may exist independently and be connected to the processor 51 via a communication bus 54. The memory 52 may also be integrated with the processor 51.

[0097] In a specific implementation, the memory 52 is used to store the data of the present invention and execute the software program of the present invention. The processor 51 can execute various functions of the air conditioner by running or executing the software program stored in the memory 52 and calling the data stored in the memory 52.

[0098] The communication interface 53 uses any transceiver-like device for communicating with other devices or communication networks, such as a Radio Access Network (RAN), a Wireless Local Area Network (WLAN), a terminal, a cloud, etc. The communication interface 53 may include a transceiver unit to implement the acquisition function.

[0099] The communication bus 54 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0100] As an example, combining Figure 6 The functions implemented by the transceiver unit 102 in the data upload device 10 are similar to Figure 7 The function of the communication interface 53 in the data upload device 10 is the same as that of the processing unit 101 in the data upload device 10. Figure 7 The function of the processor 51 in the data upload device 10 is the same as that of the storage unit 103 in the data upload device 10. Figure 7 The function of the memory 52 in is the same.

[0101] Another embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a computing device, the computing device implements any one of the data processing methods provided in the above examples.

[0102] In some embodiments, the disclosed methods may be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or on other non-transitory media or articles of manufacture.

[0103] Figure 8The figure schematically shows a conceptual partial view of a computer program product provided by an embodiment of the present invention, wherein the computer program product includes a computer program for executing a computer process on a computing device.

[0104] In one embodiment, the computer program product is provided using a signal bearing medium 410. The signal bearing medium 410 may include one or more program instructions, which when executed by one or more processors may provide the above-mentioned Figure 1 Thus, for example, reference to Figure 1 In the embodiment shown in , one or more features of S11-S14 can be undertaken by one or more instructions associated with the signal bearing medium 410. In addition, Figure 8 The program instructions in also describe example instructions.

[0105] In some examples, signal bearing medium 410 may include computer readable medium 411, such as, but not limited to, a hard drive, a compact disk (CD), a digital video disk (DVD), a digital tape, a memory, a read-only memory (ROM), or a random access memory (RAM), and the like.

[0106] In some implementations, signal bearing medium 410 may include computer recordable medium 412 such as, but not limited to, memory, read / write (R / W) CD, R / W DVD, and the like.

[0107] In some implementations, signal bearing medium 410 may include communication medium 413 such as, but not limited to, digital and / or analog communication media (eg, fiber optic cables, waveguides, wired communication links, wireless communication links, etc.).

[0108] The signal bearing medium 410 may be communicated by a wireless form of communication medium 413 (eg, a wireless communication medium conforming to the IEEE 802.41 standard or other transmission protocol). The one or more program instructions may be, for example, computer executable instructions or logic implemented instructions.

[0109] In some examples, such as for Figure 7 The described data upload device may be configured to provide various operations, functions, or actions in response to one or more program instructions via computer-readable medium 411 , computer-recordable medium 412 , and / or communication medium 413 .

[0110] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0111] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0112] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0113] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0114] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks or optical disks.

[0115] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data uploading method, characterized in that: include: Upon receiving a file upload request sent by a client for uploading a preset file, obtaining at least two upload nodes and actual performance data of each of the upload nodes; Based on the actual performance data, determining a node weight corresponding to each of the uploading nodes; wherein the node weight is inversely proportional to the node utilization corresponding to the actual performance data; Taking the upload node whose node weight is greater than the preset weight as a candidate node that can process the file upload request; The preset file is received through one or more nodes of the candidate nodes (the upload business scenario can be completed here to send the preset file to the archive server).

2. The data uploading method according to claim 1, characterized in that: The file upload request includes an Internet Protocol address; The obtaining of at least two uploading nodes and actual performance data of each of the uploading nodes includes: Obtaining an Internet Protocol address in the file upload request; Based on the Internet Protocol address, actual performance data of each uploading node in the service area corresponding to the Internet Protocol address in the current cycle is obtained.

3. The data uploading method according to claim 1, characterized in that: The determining, based on the actual performance data, a node weight corresponding to each of the uploading nodes includes: Sending the actual performance data to a preset message queue; The actual performance data in the preset message queue is processed according to a preset period using a preset stream processing framework to determine a node weight corresponding to each of the uploading nodes.

4. The data uploading method according to claim 1, characterized in that: The determining, based on the actual performance data, a node weight corresponding to each of the uploading nodes includes: The following operation is performed for each upload node: the actual performance data of the upload node is input into a weight calculation formula to perform weight calculation, and a node weight corresponding to the currently calculated upload node is obtained; wherein the process of obtaining the weight calculation formula includes: Obtain historical performance data of at least one upload node that can process file upload requests in each cycle; wherein the historical performance data includes total upload traffic, average upload rate, upload error rate, average CPU usage, and average memory usage; Based on the ratio of the total upload traffic, the average upload rate, the upload error rate, the average CPU usage rate and the average memory usage rate in all the historical performance data, determine a first coefficient of the total upload traffic, a second coefficient of the average upload rate, a third coefficient of the upload error rate, a fourth coefficient of the average CPU usage rate and a fifth coefficient of the average memory usage rate; The weight calculation formula is obtained by fitting based on the first coefficient, the second coefficient, the third coefficient, the fourth coefficient and the fifth coefficient; wherein the weight calculation formula includes: node w =-α1×traffic+α2×errorRate+α3×speed±α4×cpuUtil+α5×memoryUtil; Among them, node w represents the node weight, α1 represents the first coefficient, α2 represents the second coefficient, α3 represents the third coefficient, α4 represents the fourth coefficient, α5 represents the fifth coefficient, traffic represents the total upload traffic in the actual performance data, errorRate represents the upload error rate in the actual performance data, speed represents the average upload rate in the actual performance data, cpuUtil represents the average CPU usage in the actual performance data, and memoryUtil represents the average memory usage in the actual performance data.

5. The data uploading method according to any one of claims 1 to 4, characterized in that: The actual performance data includes total upload traffic, average upload rate, upload error rate, average CPU usage and average memory usage.

6. A data uploading device, characterized in that: include: A processing unit, configured to control the transceiver unit to obtain at least two uploading nodes and actual performance data of each of the uploading nodes when receiving a file upload request sent by a client for uploading a preset file; The processing unit is further configured to determine a node weight corresponding to each of the uploading nodes based on the actual performance data acquired by the transceiver unit; wherein the node weight is inversely proportional to the node utilization corresponding to the actual performance data; The processing unit is further configured to use an upload node whose node weight is greater than a preset weight as a candidate node that can process the file upload request; The processing unit is further configured to receive the preset file through one or more nodes among the candidate nodes.

7. The data uploading device according to claim 6, characterized in that: The file upload request includes an Internet Protocol address; The transceiver unit is specifically used to obtain the Internet Protocol address in the file upload request; The processing unit is specifically configured to obtain actual performance data of each upload node in a current cycle within a service area corresponding to the Internet Protocol address based on the Internet Protocol address obtained by the transceiver unit.

8. The data uploading device according to claim 6, characterized in that: The processing unit is specifically used to send the actual performance data to a preset message queue; The processing unit is specifically used to use a preset stream processing framework to process the actual performance data in the preset message queue according to a preset period, and determine the node weight corresponding to each of the uploading nodes.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory is used to store a computer program; and the processor is used to enable the electronic device to implement the data uploading method described in any one of claims 1 to 5 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a computing device, the computing device implements the data uploading method according to any one of claims 1 to 5.

Images