File transfer methods, devices, equipment, storage media, and software products

Abstract

This application provides a file transfer method, apparatus, device, storage medium, and program product, relating to the field of data processing. The method includes: periodically sending query requests to a task processing unit; receiving task information for each compression task and each transmission task sent by the task processing unit according to the query requests, wherein the compression task and transmission task are created by the task processing unit receiving a file to be transferred and its corresponding transmission address sent by a user's terminal device, splitting the file to be transferred into multiple sub-files to be transferred, and creating them based on the sub-files to be transferred and the transmission address; determining a target task among all tasks based on its own status parameters and the task information of each compression task and each transmission task; sending an identifier of the target task to the task processing unit and receiving the target task sent by the task processing unit according to the identifier of the target task; and executing the target task. This method solves the problem of slow file transfer speed.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a file transfer method, apparatus, device, storage medium, and program product. Background Technology

[0002] Currently, when it is necessary to remotely transfer files that occupy a lot of memory to other devices, a file transfer system is usually used.

[0003] In existing technologies, file transfer systems typically use a single node to compress an entire file before transmitting it.

[0004] However, the inventors discovered the following technical problem with the existing technology: the existing file transfer method cannot fully utilize the file compression capability and bandwidth of the file transfer system, resulting in slow file transfer speed. Summary of the Invention

[0005] This application provides a file transfer method, apparatus, device, storage medium, and program product to solve the problem of slow file transfer speed.

[0006] In a first aspect, this application provides a file transfer method applied to a proxy unit, comprising:

[0007] The system periodically sends query requests to the task processing unit; it receives task information for each compression task and each transmission task sent by the task processing unit based on the query requests. The compression and transmission tasks are created by the task processing unit receiving the file to be transmitted and its corresponding transmission address from the user's terminal device, splitting the file to be transmitted into multiple sub-files, and creating them based on the sub-files and transmission addresses; it determines the target task among all compression and transmission tasks based on its own status parameters and the task information of each compression and transmission task; it sends the target task's identifier to the task processing unit and receives the target task sent by the task processing unit based on the target task's identifier; and it executes the target task.

[0008] In one possible implementation, the target task is determined among all compression tasks and all transmission tasks based on its own state parameters, the task information of each compression task and each transmission task, including: inputting its own state parameters, the task information of each compression task and each transmission task into a pre-received target neural network model to obtain the target task.

[0009] In one possible implementation, before inputting its own state parameters, the task information of each compression task, and the task information of each transmission task into the pre-received target neural network model to obtain the target task, the method further includes: periodically sending query requests to the task processing unit at preset time intervals; receiving the task information of each compressed training task and the task information of each transmission training task sent by the task processing unit according to the query requests, wherein the compressed training tasks and transmission training tasks are created based on multiple sub-training files to be transmitted and corresponding multiple transmission training addresses, and the multiple sub-training files to be transmitted are obtained by the task processing unit receiving multiple training files to be transmitted and corresponding multiple transmission training addresses sent by the staff's terminal device and splitting the multiple training files to be transmitted; and sending the task information of each compressed training task, the task information of each transmission training task, and its own state parameters to the model training unit, so that the model training unit can process the task information of each compressed training task and the task information of each transmission training task. The system inputs its own state parameters and information into the neural network model to be optimized, obtains the target training task, and sends the identifier of the target training task to the task processing unit; receives the target training task sent by the task processing unit according to the identifier of the target training task; executes the target training task, and sends the sub-task status information of whether the target training task is completed within a preset time to the model training unit, so that the model training unit calculates the score of the neural network model to be optimized according to the sub-task status information and the main task status information sent by the task processing unit regarding whether all compression tasks and transmission tasks corresponding to a single training file to be transmitted are completed, updates the neural network model to be optimized according to the score, and obtains a new neural network model to be optimized, until the score no longer increases or the training end signal is received from the task processing unit, and the neural network model to be optimized is determined as the target neural network model. The training end signal is sent by the task processing unit when all compression training tasks and all transmission training tasks are completed.

[0010] Secondly, this application provides a task selection model processing method applied to a model training unit, comprising: receiving task information of each compressed training task and task information of each transmitted training task, along with its own state parameters, sent by any agent unit; wherein the task information of each compressed training task and task information of each transmitted training task are sent by the task processing unit according to a query request, the query request being sent by any agent unit to the task processing unit at preset intervals; the compressed training tasks and transmitted training tasks are created by the task processing unit based on multiple sub-training files to be transmitted and corresponding multiple transmission training addresses; the multiple sub-training files to be transmitted are obtained by receiving multiple training files to be transmitted and corresponding multiple transmission training addresses sent by the terminal device of the staff, and splitting the multiple training files to be transmitted; the multiple sub-training files to be transmitted correspond to each training file to be transmitted; and inputting the task information of each compressed training task and task information of each transmitted training task, along with its own state parameters, into the neural network to be optimized. The system generates a neural network model and obtains the target training task. It then sends the identifier of the target training task to the task processing unit, enabling the task processing unit to send the target training task to any agent unit based on the identifier. It receives sub-task status information from any agent unit indicating whether the target training task has been completed within a preset time. It also receives main task status information from the task processing unit indicating whether all compression and transmission tasks corresponding to a single training file to be transmitted have been completed. Based on the sub-task and main task status information, it calculates the score of the neural network model to be optimized. The system updates the neural network model to be optimized based on the score, obtaining a new model, until the score no longer increases or a training end signal is received from the task processing unit. The neural network model to be optimized is then identified as the target neural network model. The training end signal is sent by the task processing unit when all compression and transmission training tasks are completed. Finally, the target neural network model is configured to all agent units.

[0011] In one possible implementation, the score of the neural network model to be optimized is calculated based on the subtask status information and the main task status information, including: if the subtask status information indicates that the target training task is completed within a preset time, the score is increased by a first preset value; otherwise, a second preset value is subtracted. If the main task status information indicates that all compression and transmission tasks corresponding to a single training file to be transmitted are completed, the score is increased by a third preset value, where the third preset value is greater than the first and second preset values.

[0012] In one possible implementation, if the target training task is completed within a preset time, the score is increased by a first preset value; otherwise, the score is decreased by a second preset value. The implementation also includes: determining the expected completion time of the target training task based on the task information of the target training task and its own state parameters; and determining the second preset value based on the expected completion time and the preset time.

[0013] In one possible implementation, the expected completion time of the target training task is determined based on the task information of the target training task and its own state parameters, including: dividing the file size corresponding to the target training task by the compression rate of the proxy unit or the network bandwidth to obtain the expected completion time of the target training task.

[0014] In one possible implementation, the second preset value is determined based on the expected completion time and the preset time interval for the proxy unit to send query requests to the task processing unit, using the following formula:

[0015]

[0016] In the formula, y represents the second preset value, A represents a constant, T represents the expected completion time, and t represents the preset time.

[0017] Thirdly, this application provides a file transfer method applied to a task processing unit, comprising: receiving a file to be transferred and a corresponding transfer address sent by a user's terminal device; splitting the file to be transferred into multiple sub-files to be transferred; creating multiple compression tasks and multiple transfer tasks based on the sub-files to be transferred and the transfer address; receiving a query request periodically sent by any agent unit; sending task information of each compression task and task information of each transfer task to any agent unit according to the query request, so that any agent unit can determine a target task among all compression tasks and all transfer tasks based on its own status parameters and the task information of each compression task and each transfer task; receiving the identifier of the target task sent by any agent unit, and sending the target task to any agent unit according to the identifier of the target task, so that any agent unit can execute the target task.

[0018] In one possible implementation, before sending the task information of each compression task and the task information of each transmission task to any agent unit according to the query request, the method further includes: receiving multiple training files to be transmitted and corresponding multiple transmission training addresses sent by the staff's terminal device; splitting the multiple training files to be transmitted to obtain multiple sub-training files to be transmitted corresponding to each training file to be transmitted; creating multiple compression training tasks and transmission training tasks according to the multiple sub-training files to be transmitted and the corresponding multiple transmission training addresses; receiving query requests periodically sent by any agent unit at preset time intervals; and sending the task information of each compression training task and the task information of each transmission training task to any agent unit according to the query request, so that any agent unit sends the task information of each compression training task and the task information of each transmission training task, along with its own status parameters, to the model training unit, so that the model training unit sends the task information of each compression training task and the task information of each transmission training task to the model training unit. The task information and its own state parameters are input into the neural network model to be optimized, and the target training task is obtained; the identifier of the target training task sent by the model training unit is received; according to the identifier of the target training task, the target training task is sent to any agent unit so that any agent unit can execute the target training task; when all compression training tasks and all transmission training tasks are completed, a training end signal is sent to the model training unit; the sub-task status information of whether the target training task is completed within a preset time is sent to the model training unit, so that the model training unit calculates the score of the neural network model to be optimized according to the sub-task status information and the main task status information of whether all compression tasks and transmission tasks corresponding to a single training file to be transmitted are completed, sent by the task processing unit; the score is used to update the neural network model to be optimized, and a new neural network model to be optimized is obtained, until the score no longer increases or a training end signal is received, and the neural network model to be optimized is determined as the target neural network model.

[0019] Fourthly, this application provides a file transfer device, comprising: a request sending module for periodically sending query requests to a task processing unit; a first receiving module for receiving task information of each compression task and each transmission task sent by the task processing unit according to the query requests, wherein the compression task and transmission task are created by the task processing unit receiving a file to be transmitted and a corresponding transmission address sent by a user's terminal device, splitting the file to be transmitted into multiple sub-files to be transmitted, and creating them based on the sub-files to be transmitted and the transmission address; a task determination module for determining a target task among all compression tasks and all transmission tasks based on its own status parameters and the task information of each compression task and each transmission task; a second receiving module for sending an identifier of the target task to the task processing unit and receiving the target task sent by the task processing unit according to the identifier of the target task; and a task execution module for executing the target task.

[0020] Fifthly, this application provides a task selection model processing apparatus, comprising: a third receiving module, configured to receive task information of each compressed training task and task information of each transmitted training task and its own state parameters sent by any agent unit, wherein the task information of each compressed training task and task information of each transmitted training task are sent by a task processing unit according to a query request, the query request being sent by any agent unit to the task processing unit at preset intervals, the compressed training task and the transmitted training task being created by the task processing unit based on multiple sub-training files to be transmitted and corresponding multiple transmission training addresses, the multiple sub-training files to be transmitted being obtained by receiving multiple training files to be transmitted and corresponding multiple transmission training addresses sent by the terminal device of the staff, and splitting the multiple training files to be transmitted, the multiple sub-training files to be transmitted corresponding to each training file to be transmitted; a task obtaining module, configured to input the task information of each compressed training task and task information of each transmitted training task and its own state parameters into the neural network model to be optimized to obtain the target training task; and an identification sending module, configured to... The system consists of several modules: a fourth receiving module, a fifth receiving module, and a sixth receiving module. The first module sends the identifier of the target training task to the task processing unit, which then sends the target training task to any agent unit based on the identifier. The seventh module receives the sub-task status information from any agent unit indicating whether the target training task has been completed within a preset time. The eighth module receives the main task status information from the task processing unit indicating whether all compression and transmission tasks corresponding to a single training file to be transmitted have been completed. The fifth module calculates the score of the neural network model to be optimized based on the sub-task status information and the main task status information. The sixth module updates the neural network model to be optimized based on the score, obtaining a new model until the score no longer increases or a training end signal is received from the task processing unit, at which point the neural network model to be optimized is identified as the target neural network model. The training end signal is sent by the task processing unit when all compression and transmission training tasks are completed. The seventh module configures the target neural network model to all agent units.

[0021] Sixthly, this application provides a file transfer apparatus, comprising: a sixth receiving module for receiving a file to be transferred and a corresponding transfer address sent by a user's terminal device; a file splitting module for splitting the file to be transferred into multiple sub-files to be transferred; a task creation module for creating multiple compression tasks and multiple transfer tasks based on the sub-files to be transferred and the transfer address; a seventh receiving module for receiving a query request periodically sent by any agent unit; an information sending module for sending task information of each compression task and task information of each transfer task to any agent unit according to the query request, so that any agent unit can determine a target task among all compression tasks and all transfer tasks based on its own status parameters and the task information of each compression task and each transfer task; and a task sending module for receiving the identifier of the target task sent by any agent unit and sending the target task to any agent unit according to the identifier of the target task, so that any agent unit can execute the target task.

[0022] In a seventh aspect, this application provides an electronic device, including: a processor and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, causing the processor to perform a file transfer method as described in the first aspect, or a task selection model processing method as described in the second aspect, or a file transfer method as described in the third aspect.

[0023] Eighthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the file transfer method described in the first aspect, the task selection model processing method described in the second aspect, or the file transfer method described in the third aspect.

[0024] Ninthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the file transfer method described in the first aspect, or the task selection model processing method described in the second aspect, or the file transfer method described in the third aspect.

[0025] The file transfer method, apparatus, device, storage medium, and program product provided in this application periodically send query requests to the task processing unit and receive task information for compression and transmission tasks sent by the task processing unit. By combining its own status parameters and task information, it determines the target task and sends the corresponding task identifier to the task processing unit. Upon receiving the target task, it finally executes the target task. After completing all compression and transmission tasks corresponding to a file to be transferred, the purpose of sending the file to be transferred to the terminal device corresponding to the transmission address is achieved. Since each agent unit obtains the target task by combining its own status parameters and task information, it can make fuller use of the agent unit's memory, CPU (central processing unit), bandwidth, and other conditions to improve the file transfer speed. Attached Figure Description

[0026] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0027] Figure 1 This is a schematic diagram illustrating an application scenario of the file transfer method provided in the embodiments of this application;

[0028] Figure 2 Flowchart of the file transfer method provided in the embodiments of this application Figure 1 ;

[0029] Figure 3 A flowchart illustrating the task selection model processing method provided in this application embodiment;

[0030] Figure 4 Flowchart of the file transfer method provided in the embodiments of this application Figure 2 ;

[0031] Figure 5 A schematic diagram of the structure of a file transfer device provided in this application embodiment. Figure 1 ;

[0032] Figure 6 A schematic diagram of the structure of a task selection model processing device provided in an embodiment of this application;

[0033] Figure 7 A schematic diagram of the structure of a file transfer device provided in this application embodiment. Figure 2 ;

[0034] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0035] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0036] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0037] Currently, when files need to be transferred remotely or large files need to be transferred wirelessly, it is usually necessary to use a file transfer system.

[0038] In existing technologies, file transfer systems typically compress a file to be transferred as a whole and then transfer it directly as a whole. Even if there is some spare capacity in the nodes, it is usually not enough to transfer another large file at the same time. Therefore, a file usually occupies a transfer node, and the file compression capacity and bandwidth of the file transfer system cannot be fully utilized, resulting in slow file transfer.

[0039] To address the aforementioned technical problems, the inventors propose the following technical concept: by splitting the file and creating separate compression and transmission tasks for each split file, each transmission unit determines the task to be executed based on its own status parameters and task information, so as to compress and transmit all the files obtained from the split file.

[0040] Figure 1 This is a schematic diagram illustrating an application scenario of the file transfer method provided in an embodiment of this application. For example... Figure 1 In this scenario, the components include: agent unit 101, task processing unit 102, user terminal device 103, and target terminal device 104.

[0041] In the specific implementation process, the agent unit 101 can be a standalone server or a node within a server, and multiple agent units can run simultaneously. The agent unit 101 is used to query the compression and transmission tasks in the task processing unit 102, and determine the task to be executed based on its own status parameters and task information, so as to perform file compression or file transmission.

[0042] The task processing unit 102 can be a separate server or located on the same server as the agent unit 101; this application does not impose any special restrictions on this. It is used to receive the file to be transmitted and its corresponding transmission address sent by the user's terminal device 103, where the transmission address is the address of the target terminal device 104. It is also used to split the file to be transmitted into multiple sub-files to be transmitted, and to create compression tasks and transmission tasks based on the sub-files to be transmitted and their transmission addresses.

[0043] The user's terminal device 103 can be a server, computer, laptop, tablet, or mobile phone, or other device with file sending function, used to send files to the target terminal device 104 through the task processing unit 102 and the agent unit 101.

[0044] The target terminal device 104 can be a server, computer, laptop, tablet computer or mobile phone or other device with file receiving function, used to receive files sent by the user's terminal device 103 through the task processing unit 102 and the agent unit 101.

[0045] The connection between the agent unit 101, the task processing unit 102, the user's terminal device 103, and the target terminal device 104 can be either a wired connection or a wireless connection.

[0046] It is understood that the structure illustrated in the embodiments of this application does not constitute a specific limitation on the file transfer method. In other feasible embodiments of this application, the above architecture may include more or fewer components than illustrated, or combine some components, or split some components, or arrange different components, which can be determined according to the actual application scenario and is not limited here. Figure 1 The components shown can be implemented in hardware, software, or a combination of both.

[0047] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0048] Figure 2 Flowchart of the file transfer method provided in the embodiments of this application Figure 1 The execution entity of this application embodiment can be Figure 1 The agent unit 101 in the embodiment can also be a computer and / or a mobile phone, etc., and this embodiment does not impose any special restrictions on it. Figure 2 As shown, the method includes:

[0049] S201: Periodically send query requests to the task processing unit.

[0050] In this step, the periodicity can be a preset time interval, where a query request is sent to the processing unit every preset time interval. The query request can be a string composed of one or more of numbers, letters, and symbols.

[0051] S202: Receive the task information of each compression task and each transmission task sent by the task processing unit according to the query request. The compression task and transmission task are created by the task processing unit receiving the file to be transmitted and the corresponding transmission address sent by the user's terminal device, splitting the file to be transmitted into multiple sub-files to be transmitted, and creating them according to the sub-files to be transmitted and the transmission address.

[0052] In this step, the task information may include the size of the compressed file corresponding to the compression task and the size of the file to be transferred corresponding to the transfer task.

[0053] S203: Based on its own status parameters and the task information of each compression task and each transmission task, determine the target task among all compression tasks and all transmission tasks.

[0054] In this step, the self-status parameters may include one or more of the following: CPU (central processing unit) utilization, memory usage, disk space remaining, and bandwidth. The target task can be a compression task or a transfer task. Task information includes the file size of the file corresponding to the compression or transfer task.

[0055] S204: Send the identifier of the target task to the task processing unit and receive the target task sent by the task processing unit based on the identifier of the target task.

[0056] In this step, each task has a corresponding identifier, which can be one or more combinations of numbers, letters, and symbols. The receiving target task can be the file to be received, or it can be the file of the target task and its corresponding transmission address.

[0057] S205: Execute the target task.

[0058] In this step, if the target task is a compression task, the files in the target task are compressed; if the target task is a transmission task, the files corresponding to the target task are sent according to the transmission address to the target terminal device.

[0059] As can be seen from the description of the above embodiments, the embodiments of this application periodically send query requests to the task processing unit and receive task information of compression tasks and transmission tasks sent by the task processing unit. Combining its own status parameters and task information, it determines the target task, and then sends the corresponding task identifier to the task processing unit. Upon receiving the target task, it finally executes the target task. After completing all compression tasks and transmission tasks corresponding to a file to be transmitted, the purpose of sending the file to be transmitted to the terminal device corresponding to the transmission address is achieved. Since each agent unit obtains the target task by combining its own status parameters and task information, it can make fuller use of the agent unit's memory, CPU, bandwidth and other conditions to achieve the effect of improving file transmission speed.

[0060] In one possible implementation, step S203 above involves determining the target task among all compression tasks and all transmission tasks based on its own state parameters, the task information of each compression task, and the task information of each transmission task, including:

[0061] S2031: Input the self-state parameters, the task information of each compression task and each transmission task into the pre-received target neural network model to obtain the target task.

[0062] In this step, the self-state parameters, the task information of each compression task, and the task information of each transmission task can be converted to obtain numerical self-state parameters, the task information of each compression task, and the task information of each transmission task. Then, the numerical self-state parameters, the task information of each compression task, and the task information of each transmission task are input into the pre-received target neural network model to obtain the target task output by the model.

[0063] The transformation can be performed using preset production rules.

[0064] As can be seen from the description of the above embodiments, the embodiments of this application input their own state parameters, the task information of each compression task and each transmission task into the pre-received target neural network model, and determine the target task by the output of the target neural network model. This can achieve accurate positioning of the target task, find a more suitable target task for the agent unit, and thus improve the file transmission speed.

[0065] In one possible implementation, before step S2031 above, where the self-state parameters, the task information of each compression task, and each transmission task are input into the pre-received target neural network model to obtain the target task, the following further step is included:

[0066] S2030A: Periodically sends query requests to the task processing unit at preset time intervals.

[0067] This step is similar to step S201 above. The preset time can be 0.1 seconds, 1 second, 3 seconds, 1 minute, etc., and this application does not impose any special restrictions on it.

[0068] S2030B: Receives task information for each compressed training task and each transmitted training task sent by the task processing unit according to the query request. The compressed training task and the transmitted training task are created based on multiple sub-training files to be transmitted and their corresponding multiple transmitted training addresses. The multiple sub-training files to be transmitted are obtained by the task processing unit receiving multiple training files to be transmitted and their corresponding multiple transmitted training addresses sent by the staff's terminal device and splitting the multiple training files to be transmitted.

[0069] This step is similar to step S202 above, and will not be repeated here. The type of terminal device used by the staff can be any of the types of terminal devices used by the users mentioned above.

[0070] S2030C: Sends the task information of each compressed training task, the task information of each transmitted training task, and its own state parameters to the model training unit, so that the model training unit can input the task information of each compressed training task, the task information of each transmitted training task, and its own state parameters into the neural network model to be optimized, obtain the target training task, and send the identifier of the target training task to the task processing unit.

[0071] In this step, the model training unit can be a device with computing power, such as a server, CPU, computer, laptop, or tablet.

[0072] S2030D: Receives the target training task sent by the task processing unit based on the identifier of the target training task.

[0073] This step is similar to step S204 above, and will not be repeated here.

[0074] S2030E: Execute the target training task and send the sub-task status information, indicating whether the target training task has been completed within a preset time, to the model training unit. The model training unit then calculates the score of the neural network model to be optimized based on the sub-task status information and the main task status information sent by the task processing unit, indicating whether all compression and transmission tasks corresponding to a single training file to be transmitted have been completed. The model training unit updates the neural network model to be optimized based on the score to obtain a new neural network model to be optimized. This process continues until the score no longer increases or a training end signal is received from the task processing unit. The neural network model to be optimized is then identified as the target neural network model. The training end signal is sent by the task processing unit when all compression training tasks and all transmission training tasks have been completed.

[0075] In this step, the preset time is the same as the preset time in step S2030A above. The subtask status information can include either 0 or 1, or indicate in other ways whether the target training task has been completed or not within the preset time. After the model training unit determines the neural network model to be optimized as the target neural network model, it will send or deploy the target neural network model to the agent unit.

[0076] In one possible implementation, after step S2030E, step S2030F is also included: receiving the target neural network model sent by the model training unit.

[0077] As described in the above embodiments, this application embodiment periodically sends query requests to the task processing unit at preset time intervals, and receives task information of compression training tasks and transmission training tasks sent by the task processing unit according to the query requests. It inputs the task information and its own status information into the model training unit, thereby receiving the target training task sent by the task processing unit, executing the target training task, and sending the task execution status, i.e., sub-task status information, to the model training unit. This enables the model training unit to update the neural network model to be optimized according to the sub-task status information and the main task status information received from the task processing unit, thereby obtaining the target neural network model and optimizing the model. This provides a guarantee for obtaining accurate target tasks and improving file transmission speed in the future.

[0078] Figure 3 This is a flowchart illustrating the task selection model processing method provided in this application embodiment. The execution entity in this application embodiment can be the aforementioned model training unit, or it can be a computer and / or a mobile phone, etc., and this embodiment does not impose any particular limitations on it. Figure 3 As shown, the method includes:

[0079] S301: Receive task information of each compressed training task and task information of each transmitted training task, as well as its own status parameters, sent by any agent unit. The task information of each compressed training task and task information of each transmitted training task are sent by the task processing unit according to a query request. The query request is sent by any agent unit to the task processing unit at preset intervals. The compressed training task and the transmitted training task are created by the task processing unit based on multiple sub-training files to be transmitted and their corresponding multiple transmitted training addresses. The multiple sub-training files to be transmitted are obtained by receiving multiple training files to be transmitted and their corresponding multiple transmitted training addresses sent by the staff's terminal device and splitting the multiple training files to be transmitted. The multiple sub-training files to be transmitted correspond to each training file to be transmitted.

[0080] In this step, the task information and its own status parameters are described in steps S202 and S203 above.

[0081] S302: Input the task information of each compressed training task, the task information of each transmitted training task, and its own state parameters into the neural network model to be optimized, and obtain the target training task.

[0082] In this step, the neural network model to be optimized can be one of the following: feedforward neural network, feedback neural network, fully connected neural network, or convolutional neural network.

[0083] S303: Send the identifier of the target training task to the task processing unit so that the task processing unit can send the target training task to any agent unit according to the identifier of the target training task.

[0084] In this step, the identifier of the target training task can be either a representation of the target training task itself or a representation of the order in which the target training tasks were created.

[0085] S304: Receive subtask status information from any agent unit indicating whether the target training task has been completed within a preset time.

[0086] In this step, the subtask status information is as described in step S2030E above, and will not be repeated here.

[0087] S305: Receive the main task status information sent by the task processing unit regarding whether all compression and transmission tasks corresponding to a single training file to be transmitted have been completed.

[0088] In this step, the main task status information can be in a similar format to the subtask status information described above, with one or more additional digits added for differentiation.

[0089] For example, if the current subtask status is incomplete, it is represented as 0, and the main task status is incomplete, it can be represented as A0, where A serves as a distinction. It can also be represented as B0, etc. This application does not impose any restrictions on this. As another example, if the current subtask status is complete, it is represented as 1, and the main task status is incomplete, it can be represented as D1, etc.

[0090] S306: Calculate the score of the neural network model to be optimized based on the subtask status information and the main task status information.

[0091] In this step, if the subtask status is completed, the corresponding subtask score is added to the rating; if the subtask status is incomplete, the corresponding subtask score is subtracted from the rating. If the main task status is completed, the corresponding main task score is added to the rating; if the main task status is incomplete, no rating change is made, or a certain number of points may be reduced.

[0092] S307: Update the neural network model to be optimized based on the score to obtain a new neural network model to be optimized, until the score no longer increases or the training end signal is received from the task processing unit, and determine the neural network model to be optimized as the target neural network model. The training end signal is sent by the task processing unit when all compressed training tasks and all transmission training tasks are completed.

[0093] In this step, the neural network model to be optimized is updated based on the score to obtain a new neural network model to be optimized. This can be done by changing the parameters between neurons in the neural network model to be optimized based on the score, or by changing the parameters in the activation function of the neural network based on the score.

[0094] S308: Configure the target neural network model to all agent units.

[0095] In this step, configuring the target neural network model to all agent units can be done by sending the target neural network model to all agent units so that each agent unit can use the target neural network model, or by deploying the target neural network model to all agent units as part of the agent unit's operating logic.

[0096] As described in the above embodiments, this application embodiment receives task information and its own state parameters sent by the agent unit, and inputs the task information and its own state parameters into the neural network model to be optimized to obtain the target training task. The identifier of the target training task is sent to the task processing unit, so that the task processing unit allocates the target training task to the agent unit. The agent unit can execute the target training task, receive sub-task state information and main task state information sent by the agent unit, calculate the score of the neural network model to be optimized based on the sub-task state information and main task state information, and update the neural network model to be optimized based on the score to obtain a new neural network model to be optimized. This continues until the score no longer increases or a training end signal is received from the task processing unit, at which point the neural network model to be optimized is determined as the target neural network model. The target neural network model is then configured to all agent units, realizing centralized model training and decentralized model use. Each agent unit can use the target neural network model independently of the model training unit, improving the stability of file transmission. Furthermore, since the neural network model used subsequently is the target neural network model, the file transmission speed can be accelerated.

[0097] In one possible implementation, step S306 above, which calculates the score of the neural network model to be optimized based on the subtask state information and the main task state information, includes:

[0098] S3061: If the subtask status information indicates that the target training task has been completed within a preset time, the score will be increased by the first preset value; otherwise, the score will be decreased by the second preset value.

[0099] In this step, the preset time in S301 refers to the preset time. Completing the target training task within the preset time can mean completing the target training task before the agent unit sends the next query request. The first preset value is, for example, 10, 15, 20, etc. The second preset value is, for example, 1, 2, 3, etc.

[0100] S3062: If the main task status information indicates that all compression and transmission tasks corresponding to a single training file to be transmitted have been completed, then the score will be increased by a third preset value, which is greater than the first and second preset values.

[0101] In this step, the third preset value is, for example, 2000, 3000, 10000, or 20000, and this application does not impose any special restrictions on it.

[0102] As can be seen from the description of the above embodiments, the embodiments of this application change the score according to the sub-task status information and the main task status information, so that the neural network model can get a higher score when completing the target training task and transmitting a single training file to be transmitted, and a lower score when the target training task is not completed within a preset time. This enables the neural network model to iterate towards faster file transmission speed or file compression speed, resulting in a neural network model with faster file compression speed and faster file transmission speed.

[0103] In one possible implementation, before step S3061 above, if the subtask status information indicates that the target training task has been completed within a preset time, the score is increased by a first preset value; otherwise, before subtracting the second preset value, the following steps are also included:

[0104] S3060A: Determine the expected completion time of the target training task based on the task information of the target training task and its own state parameters.

[0105] In this step, the task information of the target training task and its own state parameters can be input into the pre-trained expected time calculation model to obtain the corresponding expected time.

[0106] In one possible implementation, this step determines the expected completion time of the target training task based on the task information of the target training task and its own state parameters, including:

[0107] Divide the file size corresponding to the target training task by the compression rate of the proxy unit or the network bandwidth to obtain the expected completion time of the target training task.

[0108] In this step, if the target training task is a compression training task, the file size is divided by the compression rate of the proxy unit and then added to a preset value. If the target training task is a transmission training task, the file size is divided by the network bandwidth of the proxy unit and then added to a preset value. The two preset values ​​in this step can be different to eliminate errors.

[0109] S3060B: Determine the second preset value based on the expected completion time and the preset time.

[0110] In this step, a second preset value can be obtained by adding the ratio of the expected completion time to the preset time to the preset value.

[0111] In one possible implementation, the second preset value is determined based on the expected completion time and the preset time interval for the proxy unit to send query requests to the task processing unit, using the following formula:

[0112]

[0113] In the formula, y represents the second preset value, A represents a constant, T represents the expected completion time, and t represents the preset time.

[0114] As can be seen from the description of the above embodiments, the embodiments of this application determine the expected completion time of the target task based on task information and its own state parameters, and determine a second preset value based on the expected completion time and a preset time, thereby dynamically changing the second preset value for different expected completion times, thereby achieving the effect of accelerating the model training speed.

[0115] Figure 4 Flowchart of the file transfer method provided in the embodiments of this application Figure 2 The execution entity in this embodiment can be the task processing unit described above, or it can be a computer and / or a mobile phone, etc. This embodiment does not impose any particular limitation on this. Figure 4 As shown, the method includes:

[0116] S401: Receive the file to be transferred and the corresponding transfer address sent by the user's terminal device.

[0117] In this step, the file to be transferred can be of any format, and this application does not impose any special restrictions on it.

[0118] S402: Split the file to be transferred into multiple sub-files to be transferred.

[0119] In this step, the file to be transferred is split. This can be done by splitting the file into multiple files of a fixed size, or into a fixed number of files, or by adjusting the size of the file to be transferred. The larger the file to be transferred, the larger the resulting files, or the more sub-files to be transferred. If the file size is within a preset size, splitting is not necessary, and it can be considered as splitting a single file into a single file.

[0120] 403: Create multiple compression tasks and multiple transfer tasks based on the sub-files to be transferred and the transfer address.

[0121] In this step, a compression task can be created for each sub-file to be transferred. The compressed sub-file to be transferred is combined with the transfer address to obtain a transfer task, that is, a compressed sub-file to be transferred is transferred to the transfer address as a transfer task.

[0122] S404: Receive query requests periodically sent by any agent unit.

[0123] In this step, reception can be achieved via wired or wireless means.

[0124] S405: Based on the query request, send the task information of each compression task and the task information of each transmission task to any agent unit, so that any agent unit can determine the target task among all compression tasks and all transmission tasks based on its own status parameters and the task information of each compression task and each transmission task.

[0125] In this step, upon receiving a query request, all task information can be obtained and sent to the agent unit that sent the query request.

[0126] S406: Receive the identifier of the target task sent by any agent unit, and send the target task to any agent unit according to the identifier of the target task, so that any agent unit can execute the target task.

[0127] In this step, the identifier of the target task can be received wirelessly or via a priority method.

[0128] In one possible implementation, before step S405 sends the task information of each compression task and the task information of each transmission task to any agent unit according to the query request, the method further includes:

[0129] S404A: Receives multiple training files to be transmitted and corresponding multiple transmission training addresses sent by the staff's terminal device, and splits the multiple training files to be transmitted to obtain multiple sub-training files to be transmitted corresponding to each training file to be transmitted.

[0130] In this step, for example, the training file A to be transmitted can be split into A1, A2, and A3.

[0131] This application does not restrict the format of the training files to be transmitted. The method of splitting the training files to be transmitted is as described in step S402 above, and will not be repeated here.

[0132] S404B: Create multiple compressed training tasks and transmission training tasks based on multiple sub-training files to be transmitted and their corresponding multiple transmission training addresses.

[0133] If the training file A to be transmitted in step S404A corresponds to the transmission training address C, then the transmission training addresses of A1, A2, and A3 are also C.

[0134] S404C: Receives query requests periodically sent by any agent unit at preset time intervals.

[0135] This step is similar to step S404 above, and will not be repeated here.

[0136] S404D: Based on the query request, send the task information of each compressed training task and the task information of each transmitted training task to any agent unit, so that any agent unit sends the task information of each compressed training task, the task information of each transmitted training task and its own state parameters to the model training unit, so that the model training unit inputs the task information of each compressed training task, the task information of each transmitted training task and its own state parameters into the neural network model to be optimized, and obtains the target training task.

[0137] This step is similar to step S405 above, except that the information transmission direction of the agent unit is different.

[0138] S404E: Identifier of the target training task sent by the receiving model training unit.

[0139] This step is similar to step S406 above, and will not be repeated here.

[0140] S404F: Based on the identifier of the target training task, send the target training task to any agent unit so that any agent unit can execute the target training task.

[0141] This step is similar to step S406 above, and will not be repeated here.

[0142] S404G: Sends a training end signal to the model training unit when all compression training tasks and all transmission training tasks are completed.

[0143] In this step, if there are no compression training tasks or transmission training tasks in the task list, it can be determined that the training has ended, and a training end signal can be sent to the model training unit.

[0144] S404H: Send the subtask status information of whether the target training task has been completed within a preset time to the model training unit. The model training unit calculates the score of the neural network model to be optimized based on the subtask status information and the main task status information of whether all compression tasks and transmission tasks corresponding to a single training file to be transmitted have been completed, sent by the task processing unit. The neural network model to be optimized is updated according to the score to obtain a new neural network model to be optimized. This process continues until the score no longer increases or a training end signal is received, at which point the neural network model to be optimized is determined as the target neural network model.

[0145] As described in the above embodiments, this application embodiment receives multiple training files to be transmitted and corresponding multiple transmission training addresses sent by the staff's terminal device. The training files to be transmitted are then split to obtain sub-training files corresponding to each training file. Based on these sub-training files and their corresponding transmission training addresses, compressed training tasks and transmission training tasks are created. Upon receiving a query request, the proxy unit sending the query request sends task information for each training task to the model training unit. This allows the model training unit to filter out target training tasks and receive the identifier of the target training task sent by the model training unit. Based on the identifier of the target training task, the proxy unit sends the task information to the model training unit. The processing unit sends the target training task, causing the agent unit to execute the target training task. Thus, when all compression training tasks and all transmission training tasks are completed, a training end signal is sent to the model training unit, along with the subtask status information indicating whether the target training task was completed within a preset time. This allows the model training unit to optimize the neural network model based on the subtask status information and the main task status information, resulting in the target neural network model. Since the optimization of the target neural network model is related to the task completion speed, the target neural network model can find compression or transmission tasks that make the task completion faster. Using the target neural network model can improve file transmission speed.

[0146] Figure 5 A schematic diagram of the structure of a file transfer device provided in this application embodiment. Figure 1 .like Figure 5 As shown, the file transfer device 500 includes: a request sending module 501, a first receiving module 502, a task determination module 503, a second receiving module 504, and a task execution module 505.

[0147] The request sending module 501 is used to periodically send query requests to the task processing unit.

[0148] The first receiving module 502 is used to receive the task information of each compression task and the task information of each transmission task sent by the task processing unit according to the query request. The compression task and the transmission task are created by the task processing unit receiving the file to be transmitted and the corresponding transmission address sent by the user's terminal device, splitting the file to be transmitted into multiple sub-files to be transmitted, and creating them according to the sub-files to be transmitted and the transmission address.

[0149] The task determination module 503 is used to determine the target task among all compression tasks and all transmission tasks based on its own status parameters and the task information of each compression task and each transmission task.

[0150] The second receiving module 504 is used to send the identifier of the target task to the task processing unit and receive the target task sent by the task processing unit according to the identifier of the target task.

[0151] Task execution module 505 is used to execute the target task.

[0152] In one possible implementation, the task determination module 503 is specifically used to input its own state parameters, the task information of each compression task and each transmission task into the pre-received target neural network model to obtain the target task.

[0153] In one possible implementation, the request sending module 501 is also used to periodically send query requests to the task processing unit at preset time intervals.

[0154] The first receiving module 502 is also used to receive the task information of each compressed training task and the task information of each transmission training task sent by the task processing unit according to the query request. The compressed training task and the transmission training task are created based on multiple sub-training files to be transmitted and multiple corresponding transmission training addresses. The multiple sub-training files to be transmitted are obtained by the task processing unit receiving multiple training files to be transmitted and multiple corresponding transmission training addresses sent by the terminal device of the staff and splitting the multiple training files to be transmitted.

[0155] The file transfer device 500 also includes a data sending module 506 and a status sending module 507.

[0156] The data sending module 506 is used to send the task information of each compressed training task and the task information of each transmitted training task and its own state parameters to the model training unit, so that the model training unit can input the task information of each compressed training task and the task information of each transmitted training task and its own state parameters into the neural network model to be optimized, obtain the target training task, and send the identifier of the target training task to the task processing unit.

[0157] The second receiving module 504 is also used to receive the target training task sent by the task processing unit according to the identifier of the target training task.

[0158] The status sending module 507 is used to execute the target training task and send the sub-task status information of whether the target training task is completed within a preset time to the model training unit. The model training unit calculates the score of the neural network model to be optimized based on the sub-task status information and the main task status information sent by the task processing unit regarding whether all compression and transmission tasks corresponding to a single training file to be transmitted are completed. The model training unit updates the neural network model to be optimized based on the score to obtain a new neural network model to be optimized. The process continues until the score no longer increases or a training end signal is received from the task processing unit, at which point the neural network model to be optimized is identified as the target neural network model. The training end signal is sent by the task processing unit when all compression training tasks and all transmission training tasks are completed.

[0159] Figure 6 This is a schematic diagram of the structure of a task selection model processing device provided in an embodiment of this application. Figure 6 As shown, the task selection model processing device 600 includes: a third receiving module 601, a task acquisition module 602, an identifier sending module 603, a fourth receiving module 604, a fifth receiving module 605, a scoring calculation module 606, a model determination module 607, and a model configuration module 608.

[0160] The third receiving module 601 is used to receive task information of each compressed training task and task information of each transmission training task and its own status parameters sent by any agent unit. The task information of each compressed training task and task information of each transmission training task are sent by the task processing unit according to the query request. The query request is sent by any agent unit to the task processing unit at preset intervals. The compressed training task and the transmission training task are created by the task processing unit based on multiple sub-training files to be transmitted and multiple corresponding transmission training addresses. The multiple sub-training files to be transmitted are obtained by receiving multiple training files to be transmitted and multiple corresponding transmission training addresses sent by the terminal device of the staff and splitting the multiple training files to be transmitted. The multiple sub-training files to be transmitted correspond to each training file to be transmitted.

[0161] The task acquisition module 602 is used to input the task information of each compressed training task and the task information of each transmitted training task, as well as its own state parameters, into the neural network model to be optimized, so as to obtain the target training task.

[0162] The identifier sending module 603 is used to send the identifier of the target training task to the task processing unit, so that the task processing unit can send the target training task to any agent unit according to the identifier of the target training task.

[0163] The fourth receiving module 604 is used to receive subtask status information sent by any agent unit, indicating whether the target training task has been completed within a preset time.

[0164] The fifth receiving module 605 is used to receive the main task status information sent by the task processing unit, indicating whether all compression and transmission tasks corresponding to a single training file to be transmitted have been completed.

[0165] The scoring calculation module 606 is used to calculate the score of the neural network model to be optimized based on the subtask status information and the main task status information.

[0166] The model determination module 607 is used to update the neural network model to be optimized according to the score, and obtain a new neural network model to be optimized until the score no longer increases or the training end signal is received from the task processing unit. The neural network model to be optimized is then determined as the target neural network model. The training end signal is sent by the task processing unit when all compressed training tasks and all transmission training tasks are completed.

[0167] Model configuration module 608 is used to configure the target neural network model to all agent units.

[0168] In one possible implementation, the scoring calculation module 606 is specifically used to increase the score by a first preset value if the subtask status information indicates that the target training task has been completed within a preset time; otherwise, it subtracts a second preset value. If the main task status information indicates that all compression and transmission tasks corresponding to a single training file to be transmitted have been completed, the score is increased by a third preset value, which is greater than the first and second preset values.

[0169] In one possible implementation, the task selection model processing device 600 includes: a preset value determination module 609.

[0170] The preset value determination module 609 is used to determine the expected completion time of the target training task based on the task information of the target training task and its own state parameters. Based on the expected completion time and the preset time, a second preset value is determined.

[0171] The preset value determination module 609 is specifically used to divide the file size corresponding to the target training task by the compression rate of the proxy unit or the network bandwidth to obtain the expected completion time of the target training task.

[0172] The preset value determination module 609 determines the second preset value based on the expected completion time and the preset time interval for the proxy unit to send query requests to the task processing unit. The formula is as follows:

[0173]

[0174] In the formula, y represents the second preset value, A represents a constant, T represents the expected completion time, and t represents the preset time.

[0175] Figure 7 A schematic diagram of the structure of a file transfer device provided in this application embodiment. Figure 2 .like Figure 7 As shown, the file transfer device 700 includes: a sixth receiving module 701, a file splitting module 702, a task creation module 703, a seventh receiving module 704, an information sending module 705, and a task sending module 706.

[0176] The sixth receiving module 701 is used to receive the file to be transmitted and the corresponding transmission address sent by the user's terminal device.

[0177] The file splitting module 702 is used to split the file to be transmitted into multiple sub-files to be transmitted.

[0178] The task creation module 703 is used to create multiple compression tasks and multiple transmission tasks based on the sub-files to be transmitted and the transmission address.

[0179] The seventh receiving module 704 is used to receive query requests periodically sent by any agent unit.

[0180] The information sending module 705 is used to send the task information of each compression task and the task information of each transmission task to any agent unit according to the query request, so that any agent unit can determine the target task among all compression tasks and all transmission tasks based on its own status parameters and the task information of each compression task and each transmission task.

[0181] The task sending module 706 is used to receive the identifier of the target task sent by any agent unit, and send the target task to any agent unit according to the identifier of the target task, so that any agent unit can execute the target task.

[0182] In one possible implementation, the sixth receiving module 701 is further configured to receive multiple training files to be transmitted and multiple corresponding transmission training addresses sent by the terminal device of the staff, and the file splitting module 702 is further configured to split the multiple training files to be transmitted to obtain multiple sub-training files to be transmitted corresponding to each training file to be transmitted.

[0183] The task creation module 703 is also used to create multiple compressed training tasks and transmission training tasks based on multiple sub-training files to be transmitted and their corresponding multiple transmission training addresses.

[0184] The seventh receiving module 704 is also used to receive query requests periodically sent by any agent unit at preset time intervals.

[0185] The information sending module 705 is also used to send the task information of each compressed training task and the task information of each transmitted training task to any agent unit according to the query request, so that any agent unit sends the task information of each compressed training task and the task information of each transmitted training task and its own state parameters to the model training unit, so that the model training unit inputs the task information of each compressed training task and the task information of each transmitted training task and its own state parameters into the neural network model to be optimized, and obtains the target training task.

[0186] The task sending module 706 is also used to receive the identifier of the target training task sent by the model training unit. Based on the identifier of the target training task, the target training task is sent to any agent unit so that any agent unit can execute the target training task.

[0187] The file transfer device 700 also includes a signal sending module 707 and a task status sending module 708.

[0188] The signal sending module 707 is used to send a training end signal to the model training unit when all compression training tasks and all transmission training tasks are completed.

[0189] The task status sending module 708 is used to send the sub-task status information of whether the target training task has been completed within a preset time to the model training unit. The model training unit calculates the score of the neural network model to be optimized based on the sub-task status information and the main task status information sent by the task processing unit regarding whether all compression and transmission tasks corresponding to a single training file to be transmitted have been completed. The model training unit updates the neural network model to be optimized based on the score to obtain a new neural network model to be optimized. The process continues until the score no longer increases or a training end signal is received, at which point the neural network model to be optimized is determined as the target neural network model.

[0190] The technical effects of the above-described device embodiments are the same as those of the above-described method embodiments, and will not be repeated here.

[0191] To implement the above embodiments, this application also provides an electronic device.

[0192] refer to Figure 8The diagram illustrates a structural schematic of an electronic device 800 suitable for implementing embodiments of this application. The electronic device 800 can be a terminal device or a server. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, personal digital assistants (PDAs), portable Android devices (PADs), portable media players (PMPs), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 8 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0193] like Figure 8 As shown, the electronic device 800 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 into a random access memory (RAM) 803. The RAM 803 also stores various programs and data required for the operation of the electronic device 800. The processing unit 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0194] Typically, the following devices can be connected to I / O interface 805: input devices 806 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 807 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 808 including, for example, magnetic tapes, hard disks, etc.; and communication devices 809. Communication device 809 allows electronic device 800 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 8 An electronic device 800 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0195] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 809, or installed from a storage device 808, or installed from a ROM 802. When the computer program is executed by the processing device 801, it performs the functions defined in the methods of the embodiments of this application.

[0196] It should be noted that the computer-readable storage medium described above in this application can be a computer-readable signal medium or a computer storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0197] The aforementioned computer-readable storage medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0198] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the method shown in the above embodiments.

[0199] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0200] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0201] The modules described in the embodiments of this application can be implemented in software or in hardware. The names of the units do not necessarily limit the module itself; for example, a file splitting module can also be described as a "file splitting module to be transmitted".

[0202] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0203] This application also provides a computer-readable storage medium storing computer-executable instructions. When a processor executes the computer-executable instructions, it implements the technical solution of the file transfer method or task selection model processing method in any of the above embodiments. The implementation principle and beneficial effects are similar to those of the file transfer method or task selection model processing method. Please refer to the implementation principle and beneficial effects of the file transfer method or task selection model processing method, which will not be repeated here.

[0204] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0205] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the technical solution of the file transfer method or task selection model processing method in any of the above embodiments. Its implementation principle and beneficial effects are similar to those of the file transfer method or task selection model processing method, and can be found in the implementation principle and beneficial effects of the file transfer method or task selection model processing method, which will not be repeated here.

[0206] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this application.

[0207] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0208] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk or optical disc, etc.

[0209] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0210] The aforementioned storage medium can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium accessible to general-purpose or special-purpose computers.

[0211] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. Both the processor and the storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and storage medium can exist as discrete components in an electronic device or host device.

[0212] The server in this application can also be replaced by a cluster of multiple servers.

[0213] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0214] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0215] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A file transfer method, characterized in that, Applied to the agent unit, including: Periodically send query requests to the task processing unit; The task processing unit receives task information for each compression task and task information for each transmission task sent by the task processing unit according to the query request. The compression task and the transmission task are created by the task processing unit receiving the file to be transmitted and the corresponding transmission address sent by the user's terminal device, splitting the file to be transmitted into multiple sub-files to be transmitted, and creating them according to the sub-files to be transmitted and the transmission address. The target task is obtained by inputting its own state parameters, the task information of each compression task and each transmission task into the pre-received target neural network model. Send the identifier of the target task to the task processing unit, and receive the target task sent by the task processing unit based on the identifier of the target task; Perform the target task; Before inputting the self-state parameters, the task information of each compression task, and each transmission task into the pre-received target neural network model to obtain the target task, the process further includes: The query request is periodically sent to the task processing unit at preset time intervals. The task processing unit receives task information for each compressed training task and task information for each transmitted training task sent by the task processing unit according to the query request. The compressed training task and the transmitted training task are created based on multiple sub-training files to be transmitted and multiple corresponding transmitted training addresses. The multiple sub-training files to be transmitted are obtained by the task processing unit receiving multiple training files to be transmitted and multiple corresponding transmitted training addresses sent by the terminal device of the staff and splitting the multiple training files to be transmitted. The task information of each compressed training task, the task information of each transmitted training task, and the self-state parameters are sent to the model training unit, so that the model training unit inputs the task information of each compressed training task, the task information of each transmitted training task, and the self-state parameters into the neural network model to be optimized, obtains the target training task, and sends the identifier of the target training task to the task processing unit. Receive the target training task sent by the task processing unit according to the identifier of the target training task; The target training task is executed, and the sub-task status information indicating whether the target training task has been completed within the preset time is sent to the model training unit. The model training unit then calculates the score of the neural network model to be optimized based on the sub-task status information and the main task status information sent by the task processing unit indicating whether all compression and transmission tasks corresponding to a single training file to be transmitted have been completed. The score is used to update the neural network model to be optimized, resulting in a new neural network model to be optimized. This process continues until the score no longer increases or a training end signal is received from the task processing unit. The neural network model to be optimized is then identified as the target neural network model. The training end signal is sent by the task processing unit when all compression and transmission training tasks have been completed.

2. A task selection model processing method, characterized in that, Applied to the model training unit, including: The system receives task information for each compressed training task and each transmitted training task, along with its own status parameters, sent by any agent unit. The task information for each compressed training task and each transmitted training task is sent by the task processing unit based on a query request. This query request is sent by any agent unit to the task processing unit at preset intervals. The compressed training task and the transmitted training task are created by the task processing unit based on multiple sub-training files to be transmitted and their corresponding multiple transmitted training addresses. The multiple sub-training files to be transmitted are obtained by receiving multiple training files to be transmitted and their corresponding multiple transmitted training addresses sent by the staff's terminal device, and then splitting the multiple training files to be transmitted. Each of the multiple sub-training files to be transmitted corresponds to a specific training file to be transmitted. The task information of each compressed training task, the task information of each transmitted training task, and the self-state parameters are input into the neural network model to be optimized, and the target training task is obtained. The identifier of the target training task is sent to the task processing unit, so that the task processing unit sends the target training task to any of the agent units according to the identifier of the target training task; Receive subtask status information from any of the agent units indicating whether the target training task has been completed within the preset time. Receive the main task status information sent by the task processing unit, indicating whether all compression and transmission tasks corresponding to a single training file to be transmitted have been completed; The score of the neural network model to be optimized is calculated based on the subtask status information and the main task status information. The neural network model to be optimized is updated according to the score to obtain a new neural network model to be optimized, until the score no longer increases or the training end signal is received from the task processing unit, and the neural network model to be optimized is determined as the target neural network model. The training end signal is sent by the task processing unit when all compressed training tasks and all transmission training tasks are completed. Configure the target neural network model to all agent units.

3. The method according to claim 2, characterized in that, The step of calculating the score of the neural network model to be optimized based on the subtask state information and the main task state information includes: If the subtask status information indicates that the target training task has been completed within the preset time, then the score is increased by a first preset value; otherwise, it is decreased by a second preset value. If the main task status information indicates that all compression and transmission tasks corresponding to a single training file to be transmitted have been completed, then the score will be increased by a third preset value, which is greater than the first preset value and the second preset value.

4. The method according to claim 3, characterized in that, If the subtask status information indicates that the target training task has been completed within the preset time, then the score is increased by a first preset value; otherwise, it is subtracted by a second preset value. The process also includes: Based on the task information of the target training task and its own state parameters, determine the expected completion time of the target training task; The second preset value is determined based on the expected completion time and the preset time.

5. The method according to claim 4, characterized in that, The step of determining the expected completion time of the target training task based on the task information of the target training task and the self-state parameters includes: The expected completion time of the target training task is obtained by dividing the file size corresponding to the target training task by the compression rate or network bandwidth of the proxy unit.

6. The method according to claim 4 or 5, characterized in that, The formula for determining the second preset value based on the expected completion time and the preset time interval between the proxy unit sending query requests to the task processing unit is as follows: In the formula, y represents the second preset value, A represents a constant, T represents the expected completion time, and t represents the preset time.

7. A file transfer method, characterized in that, Applied to the task processing unit, including: Receive the file to be transmitted and the corresponding transmission address sent by the user's terminal device; The file to be transmitted is split into multiple sub-files to be transmitted; Based on the sub-file to be transmitted and the transmission address, create multiple compression tasks and multiple transmission tasks; Receive query requests periodically sent by any agent unit; According to the query request, the task information of each compression task and the task information of each transmission task are sent to any of the agent units, so that any of the agent units can determine the target task among all compression tasks and all transmission tasks based on its own status parameters and the task information of each compression task and each transmission task. Receive the identifier of the target task sent by any of the agent units, and send the target task to any of the agent units according to the identifier of the target task, so that any of the agent units executes the target task; Before sending the task information of each compression task and the task information of each transmission task to any of the agent units according to the query request, the method further includes: Receive multiple training files to be transmitted and corresponding multiple transmission training addresses sent by the staff's terminal device, and split the multiple training files to be transmitted to obtain multiple sub-training files to be transmitted corresponding to each training file to be transmitted; Based on the multiple sub-training files to be transmitted and the corresponding multiple transmission training addresses, create multiple compressed training tasks and transmission training tasks. Receive query requests periodically sent by any of the agent units at preset time intervals; According to the query request, the task information of each compressed training task and the task information of each transmitted training task are sent to any of the agent units, so that any of the agent units sends the task information of each compressed training task, the task information of each transmitted training task and its own state parameters to the model training unit, so that the model training unit inputs the task information of each compressed training task, the task information of each transmitted training task and its own state parameters into the neural network model to be optimized, and obtains the target training task. Receive the identifier of the target training task sent by the model training unit; Based on the identifier of the target training task, the target training task is sent to any of the agent units so that any of the agent units can execute the target training task; When all compression training tasks and all transmission training tasks are completed, a training end signal is sent to the model training unit. The subtask status information indicating whether the target training task has been completed within the preset time is sent to the model training unit. The model training unit calculates the score of the neural network model to be optimized based on the subtask status information and the main task status information sent by the task processing unit indicating whether all compression and transmission tasks corresponding to a single training file to be transmitted have been completed. The neural network model to be optimized is updated based on the score to obtain a new neural network model to be optimized. This process continues until the score no longer increases or the training end signal is received, at which point the neural network model to be optimized is identified as the target neural network model.

8. A file transfer device, characterized in that, include: The request sending module is used to periodically send query requests to the task processing unit; The first receiving module is used to receive the task information of each compression task and the task information of each transmission task sent by the task processing unit according to the query request. The compression task and the transmission task are created by the task processing unit receiving the file to be transmitted and the corresponding transmission address sent by the user's terminal device, splitting the file to be transmitted into multiple sub-files to be transmitted, and creating them according to the sub-files to be transmitted and the transmission address. The task determination module is used to input its own state parameters, the task information of each compression task and each transmission task into the pre-received target neural network model to obtain the target task. The second receiving module is used to send the identifier of the target task to the task processing unit and receive the target task sent by the task processing unit according to the identifier of the target task. The task execution module is used to execute the target task; The request sending module is also used to periodically send query requests to the task processing unit at preset time intervals; The first receiving module is further configured to receive task information of each compressed training task and task information of each transmission training task sent by the task processing unit according to the query request. The compressed training task and the transmission training task are created based on multiple sub-training files to be transmitted and multiple corresponding transmission training addresses. The multiple sub-training files to be transmitted are obtained by the task processing unit receiving multiple training files to be transmitted and multiple corresponding transmission training addresses sent by the terminal device of the staff and splitting the multiple training files to be transmitted. The file transfer device further includes: a data sending module and a status sending module; The data sending module is used to send the task information of each compressed training task, the task information of each transmitted training task, and its own state parameters to the model training unit, so that the model training unit can input the task information of each compressed training task, the task information of each transmitted training task, and its own state parameters into the neural network model to be optimized, obtain the target training task, and send the identifier of the target training task to the task processing unit. The second receiving module is further configured to receive the target training task sent by the task processing unit according to the identifier of the target training task; The status sending module is used to execute the target training task and send the sub-task status information of whether the target training task is completed within a preset time to the model training unit. The model training unit calculates the score of the neural network model to be optimized based on the sub-task status information and the main task status information sent by the task processing unit regarding whether all compression and transmission tasks corresponding to a single training file to be transmitted are completed. The model training unit updates the neural network model to be optimized based on the score to obtain a new neural network model to be optimized. The process continues until the score no longer increases or a training end signal is received from the task processing unit, at which point the neural network model to be optimized is determined as the target neural network model. The training end signal is sent by the task processing unit when all compression training tasks and all transmission training tasks are completed.

9. A task selection model processing device, characterized in that, include: The third receiving module is used to receive task information of each compressed training task and task information of each transmission training task and its own status parameters sent by any agent unit. The task information of each compressed training task and task information of each transmission training task are sent by the task processing unit according to a query request. The query request is sent by any agent unit to the task processing unit at preset intervals. The compressed training task and the transmission training task are created by the task processing unit based on multiple sub-training files to be transmitted and multiple corresponding transmission training addresses. The multiple sub-training files to be transmitted are obtained by receiving multiple training files to be transmitted and the multiple corresponding transmission training addresses sent by the staff's terminal device and splitting the multiple training files to be transmitted. The multiple sub-training files to be transmitted correspond to each training file to be transmitted. The task acquisition module is used to input the task information of each compressed training task, the task information of each transmitted training task, and its own state parameters into the neural network model to be optimized to obtain the target training task. The identifier sending module is used to send the identifier of the target training task to the task processing unit, so that the task processing unit sends the target training task to any of the agent units according to the identifier of the target training task; The fourth receiving module is used to receive subtask status information sent by any of the agent units, indicating whether the target training task has been completed within the preset time. The fifth receiving module is used to receive the main task status information sent by the task processing unit, indicating whether all compression and transmission tasks corresponding to a single training file to be transmitted have been completed. The scoring calculation module is used to calculate the score of the neural network model to be optimized based on the subtask status information and the main task status information. The model determination module is used to update the neural network model to be optimized according to the score to obtain a new neural network model to be optimized until the score no longer increases or the training end signal is received from the task processing unit, and to determine the neural network model to be optimized as the target neural network model. The training end signal is sent by the task processing unit when all compressed training tasks and all transmission training tasks are completed. The model configuration module is used to configure the target neural network model to all agent units.

10. A file transfer device, characterized in that, include: The sixth receiving module is used to receive the file to be transmitted and the corresponding transmission address sent by the user's terminal device; The file splitting module is used to split the file to be transmitted into multiple sub-files to be transmitted. The task creation module is used to create multiple compression tasks and multiple transmission tasks based on the sub-file to be transmitted and the transmission address. The seventh receiving module is used to receive query requests periodically sent by any agent unit; The information sending module is used to send the task information of each compression task and the task information of each transmission task to any agent unit according to the query request, so that any agent unit can determine the target task among all compression tasks and all transmission tasks based on its own status parameters and the task information of each compression task and each transmission task. The task sending module is used to receive the identifier of the target task sent by any of the agent units, and send the target task to any of the agent units according to the identifier of the target task, so that any of the agent units can execute the target task; The sixth receiving module is also used to receive multiple training files to be transmitted and multiple corresponding transmission training addresses sent by the terminal device of the staff. The file splitting module is also used to split the multiple training files to be transmitted to obtain multiple sub-training files to be transmitted corresponding to each training file to be transmitted. The task creation module is also used to create multiple compressed training tasks and transmission training tasks based on multiple sub-training files to be transmitted and their corresponding multiple transmission training addresses. The seventh receiving module is also used to receive query requests periodically sent by any agent unit at preset time intervals; The information sending module is also used to send the task information of each compressed training task and the task information of each transmitted training task to any agent unit according to the query request, so that any agent unit sends the task information of each compressed training task and the task information of each transmitted training task and its own state parameters to the model training unit, so that the model training unit inputs the task information of each compressed training task and the task information of each transmitted training task and its own state parameters into the neural network model to be optimized, and obtains the target training task. The task sending module is also used to receive the identifier of the target training task sent by the model training unit, and send the target training task to any agent unit according to the identifier of the target training task, so that any agent unit can execute the target training task. The file transfer device further includes: a signal sending module and a task status sending module; The signal sending module is used to send a training end signal to the model training unit when all compression training tasks and all transmission training tasks are completed. The status sending module is used to send the sub-task status information of whether the target training task has been completed within a preset time to the model training unit. The model training unit calculates the score of the neural network model to be optimized based on the sub-task status information and the main task status information sent by the task processing unit regarding whether all compression and transmission tasks corresponding to a single training file to be transmitted have been completed. The model training unit updates the neural network model to be optimized based on the score to obtain a new neural network model to be optimized. The process continues until the score no longer increases or a training end signal is received, at which point the neural network model to be optimized is determined as the target neural network model.

11. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the file transfer method as described in claim 1, or the task selection model processing method as described in any one of claims 2 to 6, or the file transfer method as described in claim 7.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the file transfer method as described in claim 1, the task selection model processing method as described in any one of claims 2 to 6, or the file transfer method as described in claim 7.

13. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the file transfer method of claim 1, or the task selection model processing method of any one of claims 2 to 6, or the file transfer method of claim 7.

Images