Method and system for multi-source heterogeneous computing task flow decomposition

Abstract

The application discloses a kind of computing task flow decomposition method and system for multi-source heterogeneous computing power, the method includes: obtaining the computing task data sent by user end and the user data of the user end;According to the user data, the computing habit information corresponding to the user end is predicted;According to the computing habit information and the computing task data, determine task decomposition strategy;According to the task decomposition strategy, the computing task data is decomposed to obtain a plurality of computing subtasks.It can be seen that the application can realize adaptive computing task splitting based on user habit prediction, improve distributed computing efficiency and resource utilization, reduce the risk of computing delay or resource waste caused by unreasonable task decomposition.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and system for streaming decomposition of computing tasks using multi-source heterogeneous computing power. Background Technology

[0002] With the rapid growth of distributed computing and edge intelligence applications, users and platforms are increasingly focusing on improving overall computing efficiency and resource utilization through task scheduling. A key technical issue is how to avoid latency or waste caused by unreasonable task partitioning. Existing technologies typically acquire computing task data sent by the user end, decompose tasks using fixed task partitioning rules or simple heuristic algorithms, and schedule subtasks based on general resource allocation methods to support distributed execution. However, existing solutions lack predictive analysis of user computing habits and personalized adjustments to task partitioning strategies. They struggle to generate suitable subtask partitioning schemes based on user data characteristics, and common partitioning methods cannot adapt to different user usage patterns and task characteristics. This leads to increased computing latency, uneven load on some nodes, or low overall resource utilization. Unreasonable task partitioning can easily cause computing bottlenecks or resource waste, limiting the response speed, efficiency, and actual service capabilities of distributed computing platforms. Therefore, existing technologies have shortcomings that urgently need to be addressed. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a method and system for streaming decomposition of computing tasks for multi-source heterogeneous computing power, which can realize adaptive computing task splitting based on user habit prediction, improve distributed computing efficiency and resource utilization, and reduce the risk of computing delay or resource waste caused by unreasonable task decomposition.

[0004] To address the aforementioned technical problems, the first aspect of this invention discloses a method for streaming decomposition of computational tasks using multi-source heterogeneous computing power, the method comprising: Obtain the computing task data sent by the user terminal and the user data from the user terminal; Based on the user data, predict the computing habits information corresponding to the user terminal; Based on the computing habit information and the computing task data, a task decomposition strategy is determined; The computational task data is decomposed according to the task decomposition strategy to obtain multiple computational subtasks.

[0005] As an optional implementation, in the first aspect of the present invention, the computational task data includes at least one of the following: data to be predicted, prediction result specification information, data identifier to be predicted, data to be processed, data to be classified, and data acquisition channels.

[0006] As an optional implementation, in the first aspect of the present invention, the user data includes at least one of user hardware parameters, user software parameters, user communication parameters, user browsing history, and user historical calculation task records.

[0007] As an optional implementation, in the first aspect of the present invention, predicting the computing habit information corresponding to the user terminal based on the user data includes: The user data and the current time point are input into the trained LSTM neural network to obtain the prediction calculation requirements corresponding to multiple future time points; the LSTM neural network is trained using time series data that includes multiple training calculation requirements and corresponding time point calculation requirements. Based on the prediction calculation requirements corresponding to the multiple future time points, the calculation habit information corresponding to the user terminal is determined.

[0008] As an optional implementation, in the first aspect of the present invention, determining the user's corresponding computing habit information based on the prediction calculation requirements corresponding to the plurality of future time points includes: The prediction calculation requirements corresponding to each future time point are input into the trained task data generation network to obtain the prediction task data corresponding to each future time point. Calculate the data similarity between the prediction task data and the calculation task data corresponding to each future time point; Filter out all future time points whose data similarity is greater than a preset first similarity threshold to obtain multiple similar time points; Based on the prediction calculation requirements of all the similar time points, determine the calculation habit information corresponding to the user terminal.

[0009] As an optional implementation, in the first aspect of the present invention, determining the calculation habit information corresponding to the user terminal based on the prediction calculation requirements of all the similar time points includes: Calculate the intersection of the prediction calculation requirements between every two adjacent similar time points to obtain multiple sets of calculation requirements; The set of elements within the set that meet the calculation requirements is selected by filtering out the set of elements with a preset number of elements, resulting in multiple high intersection sets; Calculate the union of all the high intersection sets to obtain the computing habit information corresponding to the user terminal; the computing habit information includes at least one of computing time requirements, computing accuracy requirements, computing result format specifications, computing node requirements, and computing coherence logic requirements.

[0010] As an optional implementation, in the first aspect of the present invention, determining the task decomposition strategy based on the computing habit information and the computing task data includes: The objective function is set to minimize the number of decomposed tasks and the total computation time; the total computation time is the maximum value among the predicted computation times corresponding to all decomposed sub-tasks; the predicted computation time is obtained by inputting the task data of the decomposed sub-tasks into the trained prediction model. Setting restrictions includes: The task similarity between any two decomposed subtasks is less than a preset second similarity threshold; All the decomposed subtasks and their corresponding predicted computation times meet the requirements corresponding to the computational habit information; Based on dynamic programming algorithm and simulation environment, the computational task data is iteratively decomposed and calculated according to the objective function and the constraints until convergence, so as to obtain the optimal task decomposition strategy.

[0011] As an optional implementation, in the first aspect of the present invention, the method further includes: For each computational subtask, the optimal transmission network distance corresponding to the computational subtask is determined based on the predicted computation time corresponding to the computational subtask and the preset correspondence between time and transmission distance. Calculate the network device range centered on the current device network location and with the optimal transmission network distance as the radius; The appropriate computing node for the computing subtask is determined from all computing nodes within the network device's scope; Each computational subtask is sent to the corresponding adaptive computing node for computation.

[0012] A second aspect of this invention discloses a streaming decomposition system for computing tasks using multi-source heterogeneous computing power, the system comprising: The acquisition module is used to acquire computing task data sent by the user terminal and user data from the user terminal. The prediction module is used to predict the computing habit information corresponding to the user terminal based on the user data. The determination module is used to determine a task decomposition strategy based on the computing habit information and the computing task data; The decomposition module is used to decompose the computation task data according to the task decomposition strategy to obtain multiple computation subtasks.

[0013] As an optional implementation, in a second aspect of the present invention, the computational task data includes at least one of the following: data to be predicted, prediction result specification information, data identifier to be predicted, data to be processed, data to be classified, and data acquisition channels.

[0014] As an optional implementation, in a second aspect of the present invention, the user data includes at least one of user hardware parameters, user software parameters, user communication parameters, user browsing history, and user historical calculation task records.

[0015] As an optional implementation, in a second aspect of the invention, the specific method by which the prediction module predicts the computing habit information corresponding to the user terminal based on the user data includes: The user data and the current time point are input into the trained LSTM neural network to obtain the prediction calculation requirements corresponding to multiple future time points; the LSTM neural network is trained using time series data that includes multiple training calculation requirements and corresponding time point calculation requirements. Based on the prediction calculation requirements corresponding to the multiple future time points, the calculation habit information corresponding to the user terminal is determined.

[0016] As an optional implementation, in a second aspect of the invention, the prediction module determines the specific method by which it determines the user's corresponding calculation habit information based on the prediction calculation requirements corresponding to the plurality of future time points, including: The prediction calculation requirements corresponding to each future time point are input into the trained task data generation network to obtain the prediction task data corresponding to each future time point. Calculate the data similarity between the prediction task data and the calculation task data corresponding to each future time point; Filter out all future time points whose data similarity is greater than a preset first similarity threshold to obtain multiple similar time points; Based on the prediction calculation requirements of all the similar time points, determine the calculation habit information corresponding to the user terminal.

[0017] As an optional implementation, in a second aspect of the invention, the prediction module determines the specific method by which it determines the calculation habit information corresponding to the user terminal based on the prediction calculation requirements for all the similar time points, including: Calculate the intersection of the prediction calculation requirements between every two adjacent similar time points to obtain multiple sets of calculation requirements; The set of elements within the set that meet the calculation requirements is selected by filtering out the set of elements with a preset number of elements, resulting in multiple high intersection sets; Calculate the union of all the high intersection sets to obtain the computing habit information corresponding to the user terminal; the computing habit information includes at least one of computing time requirements, computing accuracy requirements, computing result format specifications, computing node requirements, and computing coherence logic requirements.

[0018] As an optional implementation, in a second aspect of the invention, the determining module determines the specific method of the task decomposition strategy based on the computing habit information and the computing task data, including: The objective function is set to minimize the number of decomposed tasks and the total computation time; the total computation time is the maximum value among the predicted computation times corresponding to all decomposed sub-tasks; the predicted computation time is obtained by inputting the task data of the decomposed sub-tasks into the trained prediction model. Setting restrictions includes: The task similarity between any two decomposed subtasks is less than a preset second similarity threshold; All the decomposed subtasks and their corresponding predicted computation times meet the requirements corresponding to the computational habit information; Based on dynamic programming algorithm and simulation environment, the computational task data is iteratively decomposed and calculated according to the objective function and the constraints until convergence, so as to obtain the optimal task decomposition strategy.

[0019] As an optional implementation, in a second aspect of the invention, the system is further configured to perform the following steps: For each computational subtask, the optimal transmission network distance corresponding to the computational subtask is determined based on the predicted computation time corresponding to the computational subtask and the preset correspondence between time and transmission distance. Calculate the network device range centered on the current device network location and with the optimal transmission network distance as the radius; The appropriate computing node for the computing subtask is determined from all computing nodes within the network device's scope; Each computational subtask is sent to the corresponding adaptive computing node for computation.

[0020] A third aspect of the present invention discloses another system for streaming decomposition of computational tasks using multi-source heterogeneous computing power, the system comprising: Memory containing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute some or all of the steps in the streaming decomposition method for multi-source heterogeneous computing power disclosed in the first aspect of the present invention.

[0021] The fourth aspect of the present invention discloses a computer storage medium storing computer instructions, which, when invoked, are used to execute some or all of the steps in the streaming decomposition method for multi-source heterogeneous computing power disclosed in the first aspect of the present invention.

[0022] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: This invention obtains computing task data and user data sent by the user terminal, predicts user computing habit information, and determines a task decomposition strategy accordingly. The computing task is decomposed into multiple computing sub-tasks, thereby enabling adaptive computing task splitting based on user habit prediction, improving distributed computing efficiency and resource utilization, and reducing the risk of computing delay or resource waste caused by unreasonable task decomposition. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating a method for streaming decomposition of computing tasks using multi-source heterogeneous computing power, as disclosed in an embodiment of the present invention.

[0025] Figure 2 This is a schematic diagram of a computing task streaming decomposition system for multi-source heterogeneous computing power disclosed in an embodiment of the present invention.

[0026] Figure 3 This is a schematic diagram of another computing task streaming decomposition system for multi-source heterogeneous computing power disclosed in an embodiment of the present invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0029] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0030] This invention discloses a method and system for streaming decomposition of computing tasks using multi-source heterogeneous computing power. By acquiring computing task data and user data sent by the user terminal, it predicts user computing habit information and determines a task decomposition strategy accordingly. The computing task is decomposed into multiple computing sub-tasks, thereby achieving adaptive computing task splitting based on user habit prediction. This improves distributed computing efficiency and resource utilization, and reduces the risk of computing delays or resource waste caused by unreasonable task decomposition. Detailed explanations follow.

[0031] Example 1 Please see Figure 1 , Figure 1 This is a flowchart illustrating a method for streaming decomposition of computational tasks using multi-source heterogeneous computing power, as disclosed in an embodiment of the present invention. Figure 1 The described method for streaming decomposition of computational tasks using multi-source heterogeneous computing power can be applied to data processing systems / data processing equipment / data processing servers (including local processing servers or cloud processing servers). For example... Figure 1 As shown, the method for streaming decomposition of computational tasks for multi-source heterogeneous computing power may include the following operations: 101. Obtain the computing task data sent by the user client and the user data from the user client.

[0032] Optionally, the computational task data includes at least one of the following: data to be predicted, prediction result specification information, data to be predicted identifier, data to be processed, data to be classified, and data acquisition channels.

[0033] Optionally, user data may include at least one of the following: user hardware parameters, user software parameters, user communication parameters, user browsing history, and user historical computing task records.

[0034] Optionally, the computational task data may include numerical simulation tasks, image processing tasks, machine learning training tasks, or data analysis tasks; this invention does not impose any limitations.

[0035] Optionally, the user data may include user historical operation logs, device configuration information, preference settings, or usage time patterns, which are not limited in this invention.

[0036] 102. Based on user data, predict the corresponding computing habits information of users.

[0037] 103. Determine the task decomposition strategy based on computational habits and computational task data.

[0038] Optionally, the task decomposition strategy may include task splitting granularity, subtask dependencies, or resource allocation rules, which are not limited in this invention.

[0039] 104. Decompose the computation task data according to the task decomposition strategy to obtain multiple computation subtasks.

[0040] Optionally, the multiple computational subtasks can be independent computational task data that can be executed in parallel, and this invention does not limit them.

[0041] As can be seen, the above-described embodiments of the invention obtain computing task data and user data sent by the user terminal, predict user computing habit information and determine task decomposition strategy accordingly, decompose computing tasks into multiple computing sub-tasks, thereby achieving adaptive computing task splitting based on user habit prediction, improving distributed computing efficiency and resource utilization, and reducing the risk of computing delay or resource waste caused by unreasonable task decomposition.

[0042] As can be seen, the above optional embodiments limit the details of user data content to accurately represent the relevant characteristics of users, help improve the efficiency and resource utilization of distributed computing, and reduce the risk of computing delay or resource waste caused by unreasonable task decomposition.

[0043] As an optional embodiment, the step of predicting the user's corresponding computing habits based on user data in the above steps includes: The user data and the current time point are input into the trained LSTM neural network to obtain the prediction calculation requirements for multiple future time points; Based on the prediction calculation requirements corresponding to multiple future time points, determine the corresponding calculation habit information for the user end.

[0044] Specifically, the LSTM neural network is trained using time-series data containing multiple training computational requirements and corresponding time points. Specifically, the LSTM long short-term memory neural network is trained using a time-series dataset containing multiple training user data sequences and corresponding computational requirement annotations. Optionally, the long short-term memory neural network can be a four-layer bidirectional long short-term memory network with 256 hidden units per layer, incorporating an attention mechanism, using a mean squared error loss function, and trained for 150 training epochs on 200,000 user data sequence annotations, achieving a prediction error of less than 5%. This invention is not limited to this specific approach.

[0045] As can be seen, through the above optional embodiments, by inputting user data and the current time point into the trained LSTM neural network to predict the computational requirements at multiple future time points, accurate prediction of user computational habits based on time-series learning is achieved, improving the foresight and accuracy of determining computational habit information and reducing the risk of misjudgment of habits caused by relying solely on current data.

[0046] As an optional embodiment, the step above, determining the user's corresponding computing habit information based on the prediction calculation requirements corresponding to multiple future time points, includes: The prediction calculation requirements for each future time point are input into the trained task data generation network to obtain the prediction task data for each future time point. Calculate the data similarity between the prediction task data and the calculation task data corresponding to each future time point; Filter out all future time points whose data similarity is greater than a preset first similarity threshold to obtain multiple similar time points; Based on the prediction calculation requirements of all similar time points, determine the corresponding calculation habit information for the user end.

[0047] Optionally, the task data generation network can be a variational autoencoder combined with a generative adversarial network, trained on 180,000 computationally labeled data points; this invention does not impose any limitations.

[0048] Optionally, the data similarity can be the cosine similarity of the task description vectors, which is not limited in this invention.

[0049] Optionally, the method for determining the user's corresponding calculation habit information based on the prediction calculation requirements of all similar time points can be the intersection of requirements or pattern extraction, and this invention does not limit it.

[0050] As can be seen, through the above optional embodiments, by inputting the prediction calculation requirements of future time points into the task data generation network to obtain prediction task data, calculating the similarity with the current calculation task data to filter similar time points, and determining the calculation habit information according to the prediction requirements of similar time points, the accurate extraction of habit information based on similar task prediction is realized, the relevance of habit information to the current task is improved, and the risk of habit information distortion caused by future prediction deviation is reduced.

[0051] As an optional embodiment, the step above, determining the user's corresponding calculation habit information based on the prediction calculation requirements of all similar time points, includes: Calculate the intersection of the prediction calculation requirements between any two adjacent similar time points to obtain multiple sets of calculation requirements; Filter out the sets whose number of elements is greater than the preset number of elements to obtain multiple high intersection sets; Calculate the union of all high intersection sets to obtain the corresponding computing habits information for the user.

[0052] Optionally, computational habit information may include at least one of the following: computation time requirements, computation accuracy requirements, computation result format specifications, computation node requirements, and computational coherence logic requirements.

[0053] As can be seen, through the above optional embodiments, the habit information is obtained by calculating the intersection of adjacent similar time points, filtering high intersection sets, and taking the union of the intersection sets. This achieves accurate habit feature extraction based on intersection filtering, improves the core representativeness and stability of the calculated habit information, and reduces the risk of redundancy of habit information caused by low-frequency requirement interference.

[0054] As an optional embodiment, the step of determining the task decomposition strategy based on computing habit information and computing task data in the above steps includes: The objective function is to minimize the number of decomposed tasks and the total computation time. Setting restrictions includes: The task similarity between any two decomposed subtasks is less than a preset second similarity threshold; All decomposed subtasks and their corresponding predicted computation times meet the requirements of computational habit information. Based on dynamic programming algorithms and simulation environments, the computational task data is iteratively decomposed and calculated until convergence according to the objective function and constraints to obtain the optimal task decomposition strategy.

[0055] Optionally, the total computation time is the maximum of the prediction computation times for all decomposed subtasks.

[0056] Optionally, the prediction computation time is obtained by inputting the task data of the decomposed subtasks into the trained prediction model.

[0057] Optionally, the prediction model can be a three-layer fully connected neural network, with 128 neurons in the first layer, 64 neurons in the second layer, and the third layer outputting the computation time value. This invention does not impose any limitations.

[0058] Optionally, the dynamic programming algorithm can be solved using the Bellman optimal equation, and the simulation environment is a virtual computing node cluster; this invention does not impose any limitations on this.

[0059] As can be seen, through the above optional embodiments, by setting a dual objective function of minimum number of decomposed tasks and minimum total computation time, and combining task similarity constraints and computational habit requirements, the optimal task decomposition strategy is obtained based on dynamic programming and simulation iterative calculation, thereby achieving accurate task splitting with multiple constraints, improving overall computational efficiency and habit adaptability, and reducing the risk of suboptimal decomposition due to neglect of constraints.

[0060] As an optional embodiment, the method further includes the following steps: For each computational subtask, the optimal transmission network distance for that computational subtask is determined based on the predicted computation time corresponding to that computational subtask and the preset correspondence between time and transmission distance. Calculate the network device range centered on the current device's network location, with the optimal transmission network distance as the radius; Identify the appropriate computing node for the computing subtask among all computing nodes within the network device scope; Each computational subtask is sent to the corresponding adaptive computing node for computation.

[0061] Optionally, the determination of the adapted computing node can be based on node load and / or compatibility matching, which is not limited in this invention.

[0062] Optionally, each computational subtask can be sent to the corresponding adapted computing node using a secure encrypted channel; this invention does not impose any limitations on this.

[0063] As can be seen, through the above optional embodiments, by determining the optimal transmission network distance based on the relationship between the predicted computation time and the time-transmission distance of the computation subtask, a network device range is generated with the current device as the center, and suitable computing nodes are selected from it to send the subtask for execution. This achieves accurate node allocation based on predicted time and distance, improves the low latency and reliability of task execution, and reduces the risk of computation timeout caused by excessive transmission distance.

[0064] Example 2 Please see Figure 2 , Figure 2This is a schematic diagram of a computing task streaming decomposition system for multi-source heterogeneous computing power, disclosed in an embodiment of the present invention. Figure 2 The described task streaming decomposition system for multi-source heterogeneous computing power can be applied to data processing systems / data processing equipment / data processing servers (including local processing servers or cloud processing servers). For example... Figure 2 As shown, the computational task streaming decomposition system for multi-source heterogeneous computing power may include: The acquisition module 201 is used to acquire the computing task data and user data sent by the user terminal.

[0065] The prediction module 202 is used to predict the corresponding computing habit information of the user based on user data.

[0066] The determination module 203 is used to determine the task decomposition strategy based on computing habit information and computing task data.

[0067] The decomposition module 204 is used to decompose the computation task data according to the task decomposition strategy to obtain multiple computation subtasks.

[0068] As can be seen, the above-described embodiments of the invention obtain computing task data and user data sent by the user terminal, predict user computing habit information and determine task decomposition strategy accordingly, decompose computing tasks into multiple computing sub-tasks, thereby achieving adaptive computing task splitting based on user habit prediction, improving distributed computing efficiency and resource utilization, and reducing the risk of computing delay or resource waste caused by unreasonable task decomposition.

[0069] As an optional embodiment, the computation task data includes at least one of the following: data to be predicted, prediction result specification information, data to be predicted identifier, data to be processed, data to be classified, and data acquisition channels.

[0070] As can be seen, the above optional embodiments limit the details of the task data content of the computing task data to accurately represent the characteristics of the user's computing task, help improve the efficiency and resource utilization of distributed computing, and reduce the risk of computing delay or resource waste caused by unreasonable task decomposition.

[0071] As an optional embodiment, user data includes at least one of user hardware parameters, user software parameters, user communication parameters, user browsing history, and user historical computing task records.

[0072] As can be seen, the above optional embodiments limit the details of user data content to accurately represent the relevant characteristics of users, help improve the efficiency and resource utilization of distributed computing, and reduce the risk of computing delay or resource waste caused by unreasonable task decomposition.

[0073] As an optional embodiment, the prediction module predicts the specific method by which it predicts the user's corresponding computing habits based on user data, including: User data and the current time point are input into a trained LSTM neural network to obtain prediction calculation requirements for multiple future time points; the LSTM neural network is trained using time series data that includes multiple training calculation requirements and corresponding time point calculation requirements. Based on the prediction calculation requirements corresponding to multiple future time points, determine the corresponding calculation habit information for the user end.

[0074] As can be seen, through the above optional embodiments, by inputting user data and the current time point into the trained LSTM neural network to predict the computational requirements at multiple future time points, accurate prediction of user computational habits based on time-series learning is achieved, improving the foresight and accuracy of determining computational habit information and reducing the risk of misjudgment of habits caused by relying solely on current data.

[0075] As an optional embodiment, the prediction module determines the specific method for the user's corresponding calculation habit information based on the prediction calculation requirements corresponding to multiple future time points, including: The prediction calculation requirements for each future time point are input into the trained task data generation network to obtain the prediction task data for each future time point. Calculate the data similarity between the prediction task data and the calculation task data corresponding to each future time point; Filter out all future time points whose data similarity is greater than a preset first similarity threshold to obtain multiple similar time points; Based on the prediction calculation requirements of all similar time points, determine the corresponding calculation habit information for the user end.

[0076] As can be seen, through the above optional embodiments, by inputting the prediction calculation requirements of future time points into the task data generation network to obtain prediction task data, calculating the similarity with the current calculation task data to filter similar time points, and determining the calculation habit information according to the prediction requirements of similar time points, the accurate extraction of habit information based on similar task prediction is realized, the relevance of habit information to the current task is improved, and the risk of habit information distortion caused by future prediction deviation is reduced.

[0077] As an optional embodiment, the prediction module determines the specific method for the user's corresponding calculation habit information based on the prediction calculation requirements of all similar time points, including: Calculate the intersection of the prediction calculation requirements between any two adjacent similar time points to obtain multiple sets of calculation requirements; Filter out the sets whose number of elements is greater than the preset number of elements to obtain multiple high intersection sets; Calculate the union of all high intersection sets to obtain the computational habit information corresponding to the user end; the computational habit information includes at least one of the following: computation time requirements, computation accuracy requirements, computation result format specifications, computation node requirements, and computational coherence logic requirements.

[0078] As can be seen, through the above optional embodiments, the habit information is obtained by calculating the intersection of adjacent similar time points, filtering high intersection sets, and taking the union of the intersection sets. This achieves accurate habit feature extraction based on intersection filtering, improves the core representativeness and stability of the calculated habit information, and reduces the risk of redundancy of habit information caused by low-frequency requirement interference.

[0079] As an optional embodiment, the determining module determines the specific method of the task decomposition strategy based on computing habit information and computing task data, including: The objective function is set to minimize the number of decomposed tasks and the total computation time; optionally, the total computation time is the maximum value among the predicted computation times of all decomposed subtasks; the predicted computation time is obtained by inputting the task data of the decomposed subtasks into the trained prediction model. Setting restrictions includes: The task similarity between any two decomposed subtasks is less than a preset second similarity threshold; All decomposed subtasks and their corresponding predicted computation times meet the requirements of computational habit information. Based on dynamic programming algorithms and simulation environments, the computational task data is iteratively decomposed and calculated until convergence according to the objective function and constraints to obtain the optimal task decomposition strategy.

[0080] As can be seen, through the above optional embodiments, by setting a dual objective function of minimum number of decomposed tasks and minimum total computation time, and combining task similarity constraints and computational habit requirements, the optimal task decomposition strategy is obtained based on dynamic programming and simulation iterative calculation, thereby achieving accurate task splitting with multiple constraints, improving overall computational efficiency and habit adaptability, and reducing the risk of suboptimal decomposition due to neglect of constraints.

[0081] As an optional embodiment, the system is also used to perform the following steps: For each computational subtask, the optimal transmission network distance for that computational subtask is determined based on the predicted computation time corresponding to that computational subtask and the preset correspondence between time and transmission distance. Calculate the network device range centered on the current device's network location, with the optimal transmission network distance as the radius; Identify the appropriate computing node for the computing subtask among all computing nodes within the network device scope; Each computational subtask is sent to the corresponding adaptive computing node for computation.

[0082] As can be seen, through the above optional embodiments, by determining the optimal transmission network distance based on the relationship between the predicted computation time and the time-transmission distance of the computation subtask, a network device range is generated with the current device as the center, and suitable computing nodes are selected from it to send the subtask for execution. This achieves accurate node allocation based on predicted time and distance, improves the low latency and reliability of task execution, and reduces the risk of computation timeout caused by excessive transmission distance.

[0083] Example 3 Please see Figure 3 , Figure 3 This is another computing task streaming decomposition system for multi-source heterogeneous computing power disclosed in the embodiments of the present invention. Figure 3 The described task streaming decomposition system for multi-source heterogeneous computing power is applied in data processing systems / data processing equipment / data processing servers (wherein, the server includes local processing servers or cloud processing servers). For example... Figure 3 As shown, the computational task streaming decomposition system for multi-source heterogeneous computing power may include: Memory 301 storing executable program code; Processor 302 coupled to memory 301; The processor 302 calls the executable program code stored in the memory 301 to execute the steps of the streaming decomposition method for multi-source heterogeneous computing power described in Embodiment 1.

[0084] Example 4 This invention discloses a computer read storage medium that stores a computer program for electronic data interchange, wherein the computer program causes a computer to execute the steps of the streaming decomposition method for multi-source heterogeneous computing power described in Embodiment 1.

[0085] Example 5 This invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to perform the steps of the streaming decomposition method for multi-source heterogeneous computing power described in Embodiment 1.

[0086] The foregoing has described specific embodiments of this specification; other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than those shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily have to follow the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0087] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0088] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0089] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the embodiments of this specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0090] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0091] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0092] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0093] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0094] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0095] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0096] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0097] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0098] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0099] Finally, it should be noted that the method and system for streaming decomposition of computing tasks for multi-source heterogeneous computing power disclosed in the embodiments of the present invention are merely preferred embodiments of the present invention, and are only used to illustrate the technical solutions of the present invention, not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for streaming decomposition of computational tasks using multi-source heterogeneous computing power, characterized in that, The method includes: Obtain the computing task data sent by the user terminal and the user data from the user terminal; Based on the user data, predict the computing habits information corresponding to the user terminal, including: The user data and the current time point are input into the trained LSTM neural network to obtain the prediction calculation requirements corresponding to multiple future time points; the LSTM neural network is trained using time series data that includes multiple training calculation requirements and corresponding time point calculation requirements. The prediction calculation requirements corresponding to each future time point are input into the trained task data generation network to obtain the prediction task data corresponding to each future time point. Calculate the data similarity between the prediction task data and the calculation task data corresponding to each future time point; Filter out all future time points whose data similarity is greater than a preset first similarity threshold to obtain multiple similar time points; Calculate the intersection of the prediction calculation requirements between every two adjacent similar time points to obtain multiple sets of calculation requirements; The set of elements within the set that meet the calculation requirements is selected by filtering out the set of elements with a preset number of elements, resulting in multiple high intersection sets; Calculate the union of all the high intersection sets to obtain the computing habit information corresponding to the user terminal; the computing habit information includes at least one of computing time requirements, computing accuracy requirements, computing result format specifications, computing node requirements, and computing coherence logic requirements; Based on the computational habit information and the computational task data, a task decomposition strategy is determined, including: The objective function is set to minimize the number of decomposed tasks and the total computation time; the total computation time is the maximum value among the predicted computation times corresponding to all decomposed sub-tasks; the predicted computation time is obtained by inputting the task data of the decomposed sub-tasks into the trained prediction model. Setting restrictions includes: The task similarity between any two decomposed subtasks is less than a preset second similarity threshold; All the decomposed subtasks and their corresponding predicted computation times meet the requirements corresponding to the computational habit information; Based on dynamic programming algorithm and simulation environment, the computation task data is iteratively decomposed and calculated until convergence according to the objective function and the constraints to obtain the optimal task decomposition strategy. The computational task data is decomposed according to the task decomposition strategy to obtain multiple computational subtasks.

2. The method for streaming decomposition of computational tasks for multi-source heterogeneous computing power according to claim 1, characterized in that, The computational task data includes at least one of the following: data to be predicted, prediction result specification information, data to be predicted identifier, data to be processed, data to be classified, and data acquisition channels.

3. The method for streaming decomposition of computational tasks for multi-source heterogeneous computing power according to claim 1, characterized in that, The user data includes at least one of the following: user hardware parameters, user software parameters, user communication parameters, user browsing history, and user historical calculation task records.

4. The method for streaming decomposition of computational tasks for multi-source heterogeneous computing power according to claim 1, characterized in that, The method further includes: For each computational subtask, the optimal transmission network distance corresponding to the computational subtask is determined based on the predicted computation time corresponding to the computational subtask and the preset correspondence between time and transmission distance. Calculate the network device range centered on the current device network location and with the optimal transmission network distance as the radius; The appropriate computing node for the computing subtask is determined from all computing nodes within the network device's scope; Each computational subtask is sent to the corresponding adaptive computing node for computation.

5. A streaming decomposition system for computational tasks using multi-source heterogeneous computing power, characterized in that, The system executes the streaming decomposition method for computing tasks using multi-source heterogeneous computing power as described in any one of claims 1-4, the system comprising: The acquisition module is used to acquire computing task data sent by the user terminal and user data from the user terminal. The prediction module is used to predict the computing habit information corresponding to the user terminal based on the user data. The determination module is used to determine a task decomposition strategy based on the computing habit information and the computing task data; The decomposition module is used to decompose the computation task data according to the task decomposition strategy to obtain multiple computation subtasks.

6. A computational task streaming decomposition system for multi-source heterogeneous computing power, characterized in that, The system includes: Memory containing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the streaming decomposition method for multi-source heterogeneous computing power as described in any one of claims 1-4.

Images