A computing task allocation method and device, electronic equipment and storage medium

Abstract

The application relates to the technical field of data processing, in particular to a computing task allocation method and device, electronic equipment and storage medium. The method comprises the following steps: obtaining at least one to-be-processed task of a first device, current device state information of the first device and first computing performance information of the first device; obtaining second computing performance information of a second device and link information of a target transmission link; determining a predicted encryption overhead corresponding to each to-be-processed task; determining a first predicted computing overhead corresponding to each to-be-processed task when the to-be-processed task is executed by the first device according to the first computing performance information, and determining a second predicted computing overhead corresponding to each to-be-processed task when the to-be-processed task is executed by the second device according to the second computing performance information and the link information; and based on the predicted encryption overhead, the current device state information, the first predicted computing overhead, the second predicted computing overhead and a pre-trained decision model, the method improves the task execution efficiency and the task data security.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a computing task allocation method and device, electronic equipment and a storage medium. BACKGROUND

[0002] With the increasing complexity of applications on smart mobile devices (SMDs) such as smart phones, tablets and wearable devices, the smart mobile devices face greater challenges in computing capability, storage resource capability and response speed, etc. However, due to the design goal of portability, SMDs have inherent limitations in physical size, battery capacity, heat dissipation capability, etc., which makes it difficult for SMDs to handle a variety of computing-intensive tasks and latency-sensitive computing tasks. In order to alleviate the resource bottleneck of SMDs, mobile edge computing (MEC) emerges as the times require. Through mobile edge computing, SMDs can offload computing-intensive tasks and latency-sensitive computing tasks that cannot be processed to a nearby edge server for execution. Although this approach can alleviate the resource bottleneck of SMDs, determining the task offloading decision is a very complex process. For example, offloading too many computing tasks of SMDs to the edge server can cause the edge server to overload, resulting in a significant increase in task processing latency and an increase in task discard rate, etc. For another example, when a computing task involves user privacy or contains sensitive data, if the computing task is not encrypted, it is easy to cause security problems such as leakage of private data, but encrypting and decrypting the computing task will increase a huge additional computing overhead and time delay.

[0003] Therefore, how to determine a suitable task offloading decision to cooperatively optimize the execution efficiency of the computing task and the security of the task data has become a technical problem to be solved. SUMMARY

[0004] The present application provides a computing task allocation method and device, electronic equipment and a storage medium, aiming to solve the problem of determining a suitable task offloading decision to cooperatively optimize the execution efficiency of the computing task and the security of the task data.

[0005] In a first aspect, the present application provides a computing task allocation method, which comprises:

[0006] obtaining at least one to-be-processed task of a first device, current device state information of the first device and first computing performance information of the first device;

[0007] obtaining second computing performance information of a second device and link information of a target transmission link, the target transmission link being a transmission link between the first device and the second device;

[0008] determining a predicted encryption overhead corresponding to each of the to-be-processed tasks;

[0009] determining, according to the first computing performance information, a first predicted computing overhead corresponding to execution of each of the to-be-processed tasks by the first device, and determining, according to the second computing performance information and the link information, a second predicted computing overhead corresponding to execution of each of the to-be-processed tasks by the second device;

[0010] determining, based on the predicted encryption overhead, the current device state information, the first predicted computing overhead, the second predicted computing overhead, and a pre-trained decision model, a target allocation decision for each of the to-be-processed tasks, the decision model being a double-network reinforcement learning model.

[0011] In an implementation manner, the training process of the decision model comprises:

[0012] synchronizing network parameters of an online network included in the to-be-trained double-network reinforcement learning model to a target network, the online network and the target network having the same network structure;

[0013] obtaining sample information of a preset number of interaction samples based on an experience replay pool, the sample information of the interaction samples including device state information of a sample device, an allocation decision, a standard reward, and device state information after execution of the allocation decision, the experience replay pool being configured to store sample information of a plurality of interaction samples;

[0014] inputting the sample information of the preset number of interaction samples into the target network to obtain a first predicted reward of the target network after execution of the allocation decision for each of the interaction samples;

[0015] inputting the sample information of the preset number of interaction samples into the online network to obtain a second predicted reward of the online network after execution of the allocation decision for each of the interaction samples;

[0016] determining a loss function of the to-be-trained double-network reinforcement learning model based on the first predicted reward and the second predicted reward;

[0017] determining a target evaluation index based on the loss function;

[0018] if the target evaluation index does not reach a training completion condition, adjusting network parameters of the online network;

[0019] if the target evaluation index reaches the training completion condition, determining that the training is completed;

[0020] In a case where the current iteration number reaches a first preset iteration number or the training is completed, the network parameters of the online network are synchronized to the target network, and the trained double-network reinforcement learning model after the update of the parameters is determined as the decision model.

[0021] In an implementation, the target evaluation index includes a value of the loss function or a gradient value of the loss function, and the training completion condition includes that the value of the loss function is less than a preset loss function threshold or the gradient value of the loss function is less than a preset gradient threshold.

[0022] In an implementation, the inputting of the sample information of the preset number of interaction samples into the online network to obtain the second predicted reward of the online network after the execution of the allocation decision for each interaction sample includes:

[0023] The sample information of the preset number of interaction samples is input into the online network, and the online network executes:

[0024] determination of the allocation decision for each interaction sample based on a preset training hyperparameter;

[0025] execution of the allocation decision for each interaction sample to obtain the second predicted reward corresponding to each interaction sample.

[0026] In an implementation, the execution of the allocation decision for each interaction sample to obtain the second predicted reward corresponding to each interaction sample includes:

[0027] The second predicted reward corresponding to each interaction sample is determined by using the following formula:

[0028]

[0029] wherein, is device state information of a sample device, , n t2 represents a network condition of the sample device at t moment, e t2 represents a remaining battery percentage of the sample device at t moment, represents a current CPU load occupancy rate of the sample device at t moment, and represent a delay and energy consumption of the sample device for executing a to-be-processed task locally, and represent total delay and total energy consumption consumed by the sample device for offloading the to-be-processed task to a second device, and MASK represents an encryption region selection matrix corresponding to the to-be-processed task, is an encryption delay corresponding to an encryption region of the to-be-processed task, represents a cycle of one XOR processing, for assigning the decision, for executing the assigning decision a second predicted reward, for a sample device to locally execute a computing overhead of the to-be-processed task, for a sample device to unload the to-be-processed task to a second device and execute the to-be-processed task by the second device, P2(MASK) represents a proportion of the encrypted region in the to-be-processed task, 、 and corresponding weights, 、 and corresponding weights, and corresponding weights. and P2(MASK) corresponding weights.

[0030] In an implementation, the preset training hyperparameters include a preset exploration rate;

[0031] The determining, based on the preset training hyperparameters, of the assigning decision for each of the interaction samples includes:

[0032] For each of the interaction samples, generating a random number based on device state information of a sample device corresponding to the interaction sample; and in response to the random number being not less than the preset exploration rate, determining the assigning decision for the interaction sample based on a target probability and a greedy model.

[0033] In an implementation, the determining of the predicted encryption overhead corresponding to each of the to-be-processed tasks includes:

[0034] Obtaining at least one task image of each of the to-be-processed tasks, an image label of each of the task images, a privacy level of each of the task images, and a response time tolerance of each of the task images;

[0035] Inputting each of the task images, the image label of each of the task images, the privacy level of each of the task images, and the response time tolerance of each of the task images into an encrypted region selection model to obtain an encrypted region in each of the task images, the encrypted region selection model being obtained by the second device based on a generative adversarial network in advance;

[0036] Determining a plurality of to-be-encrypted pixel points according to pixel values of each of the pixel points in the encrypted region;

[0037] Determining a predicted encryption time delay and a predicted encryption energy consumption corresponding to the plurality of to-be-encrypted pixel points as the predicted encryption overhead corresponding to each of the to-be-processed tasks.

[0038] In an implementable manner, the generative adversarial network comprises a generator and a discriminator, and the training process of the encryption region selection model comprises:

[0039] obtaining a sample labeled image dataset, the sample labeled image dataset comprising a plurality of sample images, an image label of each sample image, a privacy level of each sample image, and a response time tolerance corresponding to each sample image;

[0040] for each sample image, inputting the sample image, the privacy level of the sample image, and the response time tolerance corresponding to the sample image into the generative adversarial network, and generating, by the generator of the generative adversarial network, an encryption region corresponding to the sample image based on the sample image, the privacy level, and the response time tolerance;

[0041] performing encryption processing on each sample image based on the encryption region of each sample image to obtain an encrypted image corresponding to each sample image;

[0042] inputting each sample image and each encrypted image into the generative adversarial network, and generating, by the discriminator of the generative adversarial network, a predicted label corresponding to each sample image and each encrypted image, respectively;

[0043] determining a first loss function value of the discriminator based on the predicted label corresponding to each sample image and the image label, and the predicted label corresponding to each encrypted image;

[0044] determining a second loss function value of the generator based on each image label and the predicted label corresponding to each encrypted image;

[0045] if the first loss function value and the second loss function value are both less than a preset loss threshold or the current iteration number reaches a second preset iteration number, determining that the training of the generative adversarial network is completed, and determining the generator as the encryption region selection model; otherwise, adjusting the network parameters of the generator and the network parameters of the discriminator, and returning to perform, for each sample image, inputting the sample image, the privacy level of the sample image, and the response time tolerance corresponding to the sample image into the generative adversarial network.

[0046] In an implementable manner, the link information comprises a link bandwidth, a link channel gain, and a noise power of the target transmission link.

[0047] The determining, according to the first computing performance information, of a first predicted computing overhead corresponding to the execution of each of the to-be-processed tasks by the first device comprises:

[0048] determine, according to the first computing performance information, a first predicted time delay and a first predicted energy consumption corresponding to execution of each of the to-be-processed tasks by the first device;

[0049] perform weighted summation on the first predicted time delay and the first predicted energy consumption based on first preset weight information, to obtain a first predicted computing overhead;

[0050] determine, according to the second computing performance information and the link information, a second predicted computing overhead corresponding to execution of each of the to-be-processed tasks by the second device;

[0051] determine, according to the second computing performance information, the link bandwidth, the link channel gain and the noise power, a second predicted time delay and a second predicted energy consumption corresponding to execution of each of the to-be-processed tasks by the second device;

[0052] perform weighted summation on the second predicted time delay and the second predicted energy consumption based on second preset weight information, to obtain a second predicted computing overhead.

[0053] In an implementation manner, in a case where the target allocation decision is that each of the to-be-processed tasks is executed by the second device, the method further includes:

[0054] generate an encryption key based on a device identifier of the first device and a set of random numbers maintained by the first device;

[0055] generate a first encryption sequence and a second encryption sequence according to the encryption key and a chaotic encryption algorithm, the first encryption sequence and the second encryption sequence including a same number of parameters;

[0056] perform permutation processing on the plurality of to-be-encrypted pixel points based on the first encryption sequence, to obtain an intermediate ciphertext sequence;

[0057] perform expansion processing on the plurality of to-be-encrypted pixel points based on the second encryption sequence and the intermediate ciphertext sequence, to obtain an encrypted image of each of the to-be-processed tasks.

[0058] In an implementation manner, the target allocation decision includes a decision that each of the to-be-processed tasks is executed by the first device, and a decision that each of the to-be-processed tasks is offloaded to the second device and executed by the second device.

[0059] In a second aspect, the present application provides a computing task allocation apparatus, which includes:

[0060] a first information acquisition module, configured to acquire at least one to-be-processed task of a first device, current device state information of the first device and first computing performance information of the first device;

[0061] a second information obtaining module, configured to obtain second computing performance information of a second device and link information of a target transmission link, the target transmission link being a transmission link between the first device and the second device;

[0062] an overhead determining module, configured to determine a predicted encryption overhead corresponding to each of the to-be-processed tasks, determine a first predicted computing overhead corresponding to execution of each of the to-be-processed tasks by the first device according to the first computing performance information, and determine a second predicted computing overhead corresponding to execution of each of the to-be-processed tasks by the second device according to the second computing performance information and the link information;

[0063] a decision module, configured to determine a target allocation decision for each of the to-be-processed tasks based on the predicted encryption overhead, the current device state information, the first predicted computing overhead, the second predicted computing overhead, and a pre-trained decision model, the decision model being a double-network reinforcement learning model.

[0064] In an implementation, the apparatus further includes a model training module, configured to synchronize network parameters of an online network included in a to-be-trained double-network reinforcement learning model to a target network, the online network and the target network having the same network structure; obtain sample information of a preset number of interaction samples based on an experience replay pool, the sample information of the interaction samples including device state information of a sample device, an allocation decision, a standard reward, and device state information after execution of the allocation decision, the experience replay pool being configured to store sample information of a plurality of interaction samples; input the sample information of the preset number of interaction samples to the target network to obtain a first predicted reward of the target network after execution of the allocation decision for each of the interaction samples; input the sample information of the preset number of interaction samples to the online network to obtain a second predicted reward of the online network after execution of the allocation decision for each of the interaction samples; determine a loss function of the to-be-trained double-network reinforcement learning model based on the first predicted reward and the second predicted reward; determine a target evaluation index based on the loss function; if the target evaluation index does not reach a training completion condition, adjust the network parameters of the online network; if the target evaluation index reaches the training completion condition, determine that the training is completed; and in a case where a current iteration number reaches a first preset iteration number or the training is completed, synchronize the network parameters of the online network to the target network, and determine the to-be-trained double-network reinforcement learning model after updating of the parameters as the decision model.

[0065] In one possible implementation, the target evaluation metric includes the value of the loss function or the gradient value of the loss function; the training completion condition includes the value of the loss function being less than a preset loss function threshold or the gradient value of the loss function being less than a preset gradient threshold.

[0066] In one possible implementation, the model training module is specifically used to input sample information of a preset number of interaction samples into the online network, and the online network performs the following: determining an allocation decision for each interaction sample based on preset training hyperparameters; executing the allocation decision for each interaction sample to obtain a second predicted reward corresponding to each interaction sample.

[0067] In one possible implementation, the model training module is specifically used to determine the second predicted reward corresponding to each interaction sample using the following formula:

[0068]

[0069] in, This provides the device status information for the sample equipment. n t2 e represents the network status of the sample device at time t. t2 This represents the percentage of remaining battery power of the sample device at time t. This represents the current CPU load utilization rate of the sample device at time t. and This indicates the latency and energy consumption when the task to be processed is executed locally on the sample device. and This represents the total latency and total energy consumption consumed in offloading the task to be processed to the second device. MASK represents the encryption region selection matrix corresponding to the task to be processed. The encryption latency corresponding to the encrypted area of ​​the task to be processed. express The cycle of one XOR operation, For allocation decisions, To execute allocation decisions The second predicted reward, The computational overhead of executing the task to be processed locally on the sample device. P2(MASK) represents the computational overhead of offloading the task to be processed from the sample device to the second device and having the second device execute the task. P2(MASK) represents the proportion of the encrypted area in the task to be processed. , and They are respectively , and The corresponding weights and for a weight corresponding to P2(MASK).

[0070] In an implementable manner, the preset training hyperparameters include a preset exploration rate.

[0071] The model training module is specifically configured to, for each interaction sample, generate a random number based on device state information of a sample device corresponding to the interaction sample; and in response to the random number being not less than the preset exploration rate, determine an allocation decision for the interaction sample based on a target probability and a greedy model.

[0072] In an implementable manner, the overhead determination module is specifically configured to obtain at least one task image of each of the to-be-processed tasks, an image label of each of the task images, a privacy level of each of the task images, and a response time tolerance of each of the task images; input each of the task images, the image label of each of the task images, the privacy level of each of the task images, and the response time tolerance of each of the task images into an encryption region selection model to obtain an encryption region in each of the task images, the encryption region selection model being obtained by the second device based on a generative adversarial network; determine a plurality of to-be-encrypted pixel points according to pixel values of each of the pixels in the encryption region; and determine a predicted encryption time delay and a predicted encryption energy consumption corresponding to the plurality of to-be-encrypted pixel points as the predicted encryption overhead corresponding to each of the to-be-processed tasks.

[0073] In an implementation, the generative adversarial network comprises a generator and a discriminator, and the training process of the encryption region selection model comprises: obtaining a sample labeled image dataset, the sample labeled image dataset comprising a plurality of sample images, an image label of each sample image, a privacy level of each sample image, and a response time tolerance corresponding to each sample image; inputting, for each sample image, the sample image, the privacy level of the sample image, and the response time tolerance corresponding to the sample image into the generative adversarial network, the generator of the generative adversarial network generating an encryption region corresponding to the sample image based on the sample image, the privacy level, and the response time tolerance; performing encryption processing on each sample image based on the encryption region of each sample image to obtain an encrypted image corresponding to each sample image; inputting each sample image and each encrypted image into the generative adversarial network, the discriminator of the generative adversarial network generating a predicted label corresponding to each sample image and each encrypted image, respectively; determining a first loss function value of the discriminator based on the predicted label corresponding to each sample image and the image label, and the predicted label corresponding to each encrypted image; determining a second loss function value of the generator based on each image label and the predicted label corresponding to each encrypted image; if the first loss function value and the second loss function value are both less than a preset loss threshold or a current iteration number reaches a second preset iteration number, determining that the training of the generative adversarial network is completed, and determining the generator as the encryption region selection model; otherwise, adjusting network parameters of the generator and network parameters of the discriminator, and returning to perform inputting, for each sample image, the sample image, the privacy level of the sample image, and the response time tolerance corresponding to the sample image into the generative adversarial network.

[0074] In a third aspect, a computer program product is provided, which includes a memory and a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the program, the computer program product implements the method for allocating computing tasks.

[0075] In a fourth aspect, a storage medium containing computer executable instructions is provided. When the computer executable instructions are executed by a computer processor, the computer executable instructions are used to execute the method for allocating computing tasks.

[0076] The method for allocating a computing task provided in the embodiments of the present application comprises the following steps: obtaining at least one to-be-processed task of a first device, current device state information of the first device, and first computing performance information of the first device; obtaining second computing performance information of a second device and link information of a target transmission link, the target transmission link being a transmission link between the first device and the second device; determining a predicted encryption overhead corresponding to each to-be-processed task; determining a first predicted computing overhead corresponding to each to-be-processed task when the to-be-processed task is executed by the first device according to the first computing performance information, and determining a second predicted computing overhead corresponding to each to-be-processed task when the to-be-processed task is executed by the second device according to the second computing performance information and the link information; and determining a target allocation decision for each to-be-processed task based on the predicted encryption overhead, the current device state information, the first predicted computing overhead, the second predicted computing overhead, and a pre-trained decision model, the decision model being a double-network reinforcement learning model. The encryption overhead of the to-be-processed task is associated with the allocation decision, and the computing amount and the transmission overhead caused by the encryption operation are considered when the computing overhead corresponding to the to-be-processed task is determined, so that the privacy protection cost of the to-be-processed task is taken into account, and the target allocation decision determined is more accurate. Moreover, the double-network structure of the double-network reinforcement learning model can make the decision process more stable and the convergence better, the decision model obtained by pre-training the double-network reinforcement learning model is used to make the task allocation decision, the computing overhead of the to-be-processed task can be comprehensively considered, the computing performance of the device, the privacy of the to-be-processed task, the computing cost and the encryption cost are balanced, and the target allocation decision generated can realize collaborative optimization of the execution efficiency of the computing task and the security of the task data.

[0077] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0078] The above and other objects, features and advantages of the present exemplary embodiments will become more apparent from the following detailed description read in conjunction with the accompanying drawings, in which like reference numerals refer to like elements. Several embodiments of the present application are illustrated in the drawings, in which:

[0079] In the drawings, identical or corresponding reference numerals indicate identical or corresponding parts.

[0080] Figure 1 An implementation flowchart of a method for allocating a computing task provided in the embodiments of the present application is shown;

[0081] Figure 2 A training flowchart of an encryption region selection model provided in the embodiments of the present application is shown;

[0082] Figure 3 A training flowchart of the decision model provided by the embodiments of the present application is shown.

[0083] Figure 4 A task encryption flowchart provided by the embodiments of the present application is shown.

[0084] Figure 5 A structural diagram of the computing task allocation method provided by the embodiments of the present application is shown.

[0085] Figure 6 A structural diagram of an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0086] In order to make the objectives, characteristics and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0087] Since the SMDs cannot handle the computationally intensive tasks and the latency sensitive computing tasks, the edge server is used to execute the tasks, which can easily cause the task processing time delay to increase greatly and the task discard rate to increase. Therefore, in order to solve the problem of determining a suitable task offloading decision to cooperatively optimize the execution efficiency of the computing task and the security of the task data, the present application provides a computing task allocation method, device, electronic device and storage medium. The computing task allocation method provided by the present application can be applied to a first device and a second device. The first device is an SMDs device, such as a mobile phone, a tablet computer, a telephone watch and a notebook computer, etc. The second device can include an edge server, a virtual machine and an edge computing platform, etc.

[0088] The technical solutions of the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.

[0089] Figure 1 An implementation flowchart of the computing task allocation method provided by the embodiments of the present application is shown, as shown in Figure 1 The method comprises the following steps.

[0090] S101, obtaining at least one to-be-processed task of a first device, current device state information of the first device and first computing performance information of the first device.

[0091] In the present application, the to-be-processed task is a computing task of a task that needs to be calculated and / or consumes computing resources. For example, the to-be-processed task can include an image processing task, a video processing task, a data migration task, and a model rendering task, etc.

[0092] In the present application, N computing tasks that need to be processed in the first device within a preset period can be obtained as to-be-processed tasks every interval of the preset period. The value of N can be an integer greater than or equal to 1. The preset period can be set to 1 minute or 2 minutes, etc. For example, at time t, it can be determined that there are N computing tasks that need to be processed in the first device within a preset period with time t as the starting time point, and then the N computing tasks can all be constructed as to-be-processed tasks to construct a to-be-processed task set , Set , wherein, are to-be-processed tasks, and f can be a device number representing the first device. Each to-be-processed task can include a plurality of task parameters. The task parameters can include an input data amount parameter, a required resource parameter, an output data amount parameter, and a maximum tolerable time delay. For example, the i-th to-be-processed task may include an input data amount parameter , a required resource parameter , an output data amount parameter , and a maximum tolerable time delay . , wherein the input data amount parameter specifically represents the size of the input data amount required to complete the i-th to-be-processed task ; the required resource parameter specifically represents the computing resource required to complete the i-th to-be-processed task , such as the number of GPUs and the size of storage resources required, etc.; the output data amount parameter specifically represents the size of the data amount output after the i-th to-be-processed task is completed, for example, if the second device processes the i-th to-be-processed task , the output data amount parameter represents the size of the result data amount that needs to be returned to the first device after the second device processes the i-th to-be-processed task ; and the maximum tolerable time delay specifically represents that the time delay of completing the i-th to-be-processed task should not exceed time .

[0093] The current device state information of the first device can include network state information of the first device at the current time, power information of the first device, load information of the first device, and task information of a to-be-processed task of the first device. For example, when the current time is t, the current device state information of the first device at the time t can be represented as , wherein the current device state information includes network state information of the first device at the time t , power information of the first device , load information of the first device , and task information of a to-be-processed task of the first device . The network state information may specifically be the network signal strength of the first device at the time t; the power information may specifically be the remaining power percentage of the first device at the time t; the load information may specifically be the CPU load occupancy rate of the first device at the time t; and the task information of the to-be-processed task may be N to-be-processed tasks that the first device needs to complete in a preset period with the time t as the starting time .

[0094] The first computing performance information of the first device can include the CPU frequency of the first device and the device power. For example, the working state power consumption of the first device in each CPU cycle can be taken as the device power.

[0095] S102, obtaining second computing performance information of a second device and link information of a target transmission link.

[0096] The target transmission link is a transmission link between the first device and the second device. The target transmission link can be a wireless transmission link, for example, a cellular network link or a Bluetooth transmission link, etc. The target transmission link can also be a wired transmission link, for example, an Ethernet link or a fiber transmission link, etc. The target transmission link can also be a content distribution network link or an edge gateway link, etc.

[0097] In the present application, the second computing performance information of the second device can include the CPU frequency of the second device and the device power of the second device. The link information of the target transmission link can include the link bandwidth, the link channel gain, the noise power and the link transmission power of the target transmission link, etc.

[0098] S103, determining a predicted encryption overhead corresponding to each of the to-be-processed tasks.

[0099] Since pending tasks may contain private information, if these tasks are to be offloaded from one device to a second device—for example, if the task contains facial images (which are typically considered private information)—encryption is necessary to protect the security of this private content and prevent its leakage during the offloading process. Encryption consumes computational resources and increases the processing latency of the task. The encryption overhead of the task can be used to reflect the computational resources consumed and the latency of encrypting the task.

[0100] S104, determine the first predicted computational cost corresponding to each of the tasks to be processed when executing each of the tasks to be processed through the first device based on the first computational performance information, and determine the second predicted computational cost corresponding to each of the tasks to be processed when executing each of the tasks to be processed through the second device based on the second computational performance information and the link information.

[0101] In this application, the first predictive computational overhead can reflect the energy consumption and latency of the first device processing each task to be processed. The second predictive computational overhead can reflect the energy consumption and latency of the second device processing each task to be processed.

[0102] S105, based on the predicted encryption overhead, the current device state information, the first predicted computation overhead, the second predicted computation overhead, and the pre-trained decision model, determine the target allocation decision for each of the tasks to be processed.

[0103] The decision-making model is a dual-network reinforcement learning model. A dual-network reinforcement learning model consists of two networks with identical structures.

[0104] In this application, a decision model is pre-trained on a second device using a dual-network reinforcement learning model to be trained. After obtaining the decision model, the second device can distribute it to a first device to determine target allocation decisions for each of the tasks to be processed. Alternatively, the second device may choose not to distribute the decision model to the first device. The first device can send the prediction encryption overhead, the current device state information, the first prediction computation overhead, and the second prediction computation overhead to the second device, whereby the decision model in the second device determines the target allocation decisions for each of the tasks to be processed.

[0105] In this application, the target allocation decision for the tasks to be processed can be a decision to have the first device execute each of the tasks, or a decision to offload the tasks to be processed to the second device and have the second device execute each of the tasks. For example, the target allocation decision for the tasks to be processed... The value can be 0 or 1. At that time, the set of tasks to be processed the i-th to-be-processed task complete the processing locally at the first device; when the set of to-be-processed tasks the i-th to-be-processed task offload to the second device, and complete the processing of the to-be-processed tasks by the second device.

[0106] In the present application, if the network status, power, and current CPU load occupancy rate of the first device change, such that the first device cannot bear the processing operations of the to-be-processed tasks, the task allocation method provided in the present application can determine, as the target allocation decision, the decision of offloading each to-be-processed task to the second device and executing each to-be-processed task by the second device through its own computing resources. For example, in the case where the network status of the first device is good, the remaining power of the first device is sufficient, and the current CPU load occupancy rate is low, each to-be-processed task can be offloaded to the second device, and the second device can process each to-be-processed task to obtain the corresponding processing result and return the processing result to the first device. The first device can reduce the task processing delay and energy consumption by offloading the to-be-processed tasks to the second device and executing the to-be-processed tasks by the second device, and at the same time, the computing resources of the first device are saved, and the task processing environment of the first device is improved. If the network status of the first device is poor, in this case, offloading the to-be-processed tasks to the second device can cause huge delay and energy consumption. In order to reduce the overall delay and energy consumption of task processing, each to-be-processed task can be processed by the first device. The first device can include various computing application services, such as database services, big data processing services, and content storage services. If the target allocation decision is to execute each to-be-processed task by the first device, the first device can process each to-be-processed task through various computing application services. If the network status of the first device is poor, and the remaining power of the first device is small and / or the current CPU load occupancy rate of the first device is high, in this case, the task allocation method of the present application can determine the allocation decision that can make the overall delay and energy consumption of task processing lower as the target allocation decision. For example, if the overall delay and energy consumption of executing each to-be-processed task locally at the first device is lower than that of offloading each to-be-processed task to the second device and executing each to-be-processed task by the second device, the decision of executing each to-be-processed task at the first device can be taken as the target allocation decision, otherwise, the decision of offloading each to-be-processed task to the second device and executing each to-be-processed task by the second device can be taken as the target allocation decision.

[0107] ​The method for allocating a computing task provided in the embodiments of the present application comprises the following steps: obtaining at least one to-be-processed task of a first device, current device state information of the first device, and first computing performance information of the first device; obtaining second computing performance information of a second device and link information of a target transmission link, the target transmission link being a transmission link between the first device and the second device; determining a predicted encryption overhead corresponding to each to-be-processed task; determining a first predicted computing overhead corresponding to each to-be-processed task when the to-be-processed task is executed by the first device according to the first computing performance information, and determining a second predicted computing overhead corresponding to each to-be-processed task when the to-be-processed task is executed by the second device according to the second computing performance information and the link information; and determining a target allocation decision for each to-be-processed task based on the predicted encryption overhead, the current device state information, the first predicted computing overhead, the second predicted computing overhead, and a pre-trained decision model, the decision model being a double-network reinforcement learning model. The encryption overhead of the to-be-processed task is associated with the allocation decision in the present application, and the computing amount and the transmission overhead caused by the encryption operation are considered when the computing overhead corresponding to the to-be-processed task is determined, so that the privacy protection cost of the to-be-processed task is taken into account, and the target allocation decision determined is more accurate. Moreover, the double-network structure of the double-network reinforcement learning model can make the decision process more stable and have better convergence, and the decision model obtained by pre-training the double-network reinforcement learning model is used for task allocation decision in the present application, which can comprehensively consider the computing overhead of the to-be-processed task, balance the computing performance of the device, the privacy of the to-be-processed task, the computing cost and the encryption cost, so that the target allocation decision generated can realize collaborative optimization of the execution efficiency of the computing task and the security of the task data.

[0108] In a possible implementation, the determining of the predicted encryption overhead corresponding to each to-be-processed task can comprise steps A1-A4.

[0109] Step A1: obtaining at least one task image of each to-be-processed task, an image label of each task image, a privacy level of each task image, and a response time tolerance of each task image.

[0110] In the present application, the to-be-processed task can be an image processing task or a computing task containing an image. The image of the to-be-processed task can be taken as a task image. The image label of the task image is used to identify the content of the task image. For example, if the task image is a face image of a person, the image label can be the name or other identity information of the person. The image label of the task image can be represented in the form of a number or a string. The privacy level of the task image is used to represent the privacy degree of the content of the task image, and the higher the privacy level, the higher the privacy degree of the content of the task image. The response time tolerance corresponding to the task image refers to that the processing of the task image should not exceed the response time tolerance.

[0111] Step A2, inputting each of the task images, the image label of each of the task images, the privacy level of each of the task images and the response time tolerance of each of the task images into an encryption region selection model to obtain an encryption region in each of the task images.

[0112] The encryption region selection model is pre-trained based on a generative adversarial network for the second device. In this application, the encryption region selection model can be pre-trained based on a generative adversarial network on the second device, and then distributed to the first device for encryption processing of the to-be-processed task.

[0113] The generative adversarial network can include a generator and a discriminator. Figure 2 A training process schematic diagram of the encryption region selection model provided by the embodiments of the application is shown as follows, Figure 2 The training process of the encryption region selection model includes:

[0114] S201, obtaining a sample labeled image dataset.

[0115] The sample labeled image dataset includes a plurality of sample images, an image label of each of the sample images, a privacy level of each of the sample images and a response time tolerance corresponding to each of the sample images.

[0116] The image label of the sample image is used to identify the content of the sample image. For example, if the sample image is a face image of Li, the image label can be the name or other identity information of Li; if the sample image is an animal image, the image label can be the scientific name of the animal. The image label of the sample image can be represented by numbers or strings, for example, if the image label is the name of Li, a string uniquely associated with the name of Li can be used as the image label.

[0117] The privacy level of the sample image is used to represent the privacy degree of the content of the sample image. The higher the privacy level, the higher the privacy degree of the content of the sample image.

[0118] The response time tolerance corresponding to the sample image means that the processing of the sample image should not exceed the response time tolerance.

[0119] In this application, the first device can construct a sample labeled image dataset and send the encrypted sample labeled image dataset to the second device. Alternatively, the second device can actively initiate a sample labeled image dataset acquisition request to the first device at a preset time interval, and then the first device sends the encrypted sample labeled image dataset to the second device according to the sample labeled image dataset acquisition request. The second device can decrypt the encrypted sample labeled image dataset to obtain a decrypted sample labeled image dataset, which is used to train or update the encryption region selection model.

[0120] In the present application, the second device can only obtain a plurality of sample images with image labels from the first device. The second device can process the plurality of sample images with image labels to construct a sample labeled image dataset. For example, for each sample image P, the second device can randomly sample J values t1, t2, …, t low high ] within the common response time tolerance range [t J low is the lowest tolerance value of the common response time tolerance, and t high is the highest tolerance value of the common response time tolerance. Based on the J response time tolerances and the three privacy levels v = 1, 2, 3, the sample image P can be expanded to 3xJ training samples by assigning different privacy levels and response time tolerances to the sample image P.

[0121] S202, for each sample image, input the sample image, the privacy level of the sample image, and the response time tolerance corresponding to the sample image into the generative adversarial network, and the generator of the generative adversarial network generates an encrypted region corresponding to the sample image based on the sample image, the privacy level, and the response time tolerance.

[0122] In the present application, the generator of the generative adversarial network can generate an encrypted region selection matrix corresponding to the sample image based on the sample image, the privacy level, and the response time tolerance. The number of rows of the encrypted region selection matrix is the same as the height of the sample image, and the number of columns of the encrypted region selection matrix is the same as the width of the sample image. The height of the sample image P is H, the width of the sample image P is W, the image matrix of the three-channel image of the sample image P is HxWx3, the privacy level of the sample image P is v, and the response time tolerance of the sample image P is t. The image matrix HxWx3 of the sample image P, the privacy level v, and the response time tolerance t can be input into the generative adversarial network, and the generator of the generative adversarial network generates an encrypted region selection matrix Mask with H rows and W columns. Each element of the encrypted region selection matrix Mask has a value of 0 or 1, indicating no encryption or encryption, respectively. For example, if the value of the element of the first row and the first column of the encrypted region selection matrix Mask is 0, it means that the pixel point at the corresponding position in the sample image needs to be encrypted; if the value of the element of the first row and the first column of the encrypted region selection matrix Mask is 1, it means that the pixel point at the corresponding position in the sample image does not need to be encrypted. Based on the encrypted region selection matrix, the smallest image region in the sample image that includes each pixel point that needs to be encrypted can be taken as the encrypted region. Alternatively, the encrypted region selection matrix can also be taken as the encrypted region, and the encrypted region selection matrix can be used to realize precise encryption of the private content of the sample image.

[0123] ​​S203, performing encryption processing on each of the sample images based on the encryption region of each of the sample images, to obtain an encrypted image corresponding to each of the sample images.

[0124] In the present application, an encryption key can be generated based on the device identifier of the first device and the set of random numbers maintained by the first device; and each pixel point of each of the sample images that needs to be encrypted is encrypted according to the encryption key and a chaotic encryption algorithm, to obtain an encrypted image corresponding to each of the sample images.

[0125] S204, inputting each of the sample images and each of the encrypted images into the generative adversarial network, and the discriminator of the generative adversarial network generating a prediction label corresponding to each of the sample images and each of the encrypted images respectively.

[0126] In the present application, in the training process, for each of the encrypted images corresponding to each sample image, the discriminator of the generative adversarial network can be used to generate a prediction label corresponding to each sample image and each encrypted image respectively, that is, the discriminator is used to detect whether the encryption region selection matrix generated by the generator can make the encrypted image unable to be accurately identified by the discriminator. The prediction label is used to represent the image content.

[0127] S205, determining a first loss function value of the discriminator based on the prediction label corresponding to each of the sample images and the image label, and the prediction label corresponding to each of the encrypted images.

[0128] In the present application, the first loss function value of the discriminator can be determined based on the prediction label corresponding to each of the sample images and the image label, and the prediction label corresponding to each of the encrypted images, using the following formula:

[0129]

[0130] wherein, The first loss function value of the discriminator is N, the number of pixel points of the sample image, p i is the i-th sample image, P is a sample labeled image data set including each of the sample images, P' is a set composed of each of the encrypted images, y i is the sample label of p i , and y i ' is the prediction label of p i .

[0131] In the present application, the first loss function value of the discriminator can also be determined based on the prediction label corresponding to each of the sample images and the image label, and the prediction label corresponding to each of the encrypted images, using the following formula:

[0132]

[0133] wherein, characterizing the i-th sample image the corresponding encrypted region selection matrix Mask, characterizing the i-th sample image the predicted label of the i-th sample image, characterizing the i-th sample image the predicted label of the corresponding encrypted image.

[0134] S206, determining the second loss function value of the generator based on each of the image labels and the predicted labels corresponding to each of the encrypted images.

[0135] In the present application, the second loss function value of the generator can be determined based on each of the image labels and the predicted labels corresponding to each of the encrypted images by using the following formula:

[0136]

[0137] wherein, is the second loss function value of the generator, denotes the sum of all elements of the encrypted region selection matrix Mask, is the weight corresponding to the privacy level v, is the weight corresponding to the response time tolerance t. The weight corresponding to the privacy level v and the weight corresponding to the response time tolerance t can be pre-set according to the importance of the privacy content and the time tolerance, for example, if the weight corresponding to the privacy level v and the weight corresponding to the response time tolerance t are set to 0.4 and 0.6 respectively, or 0.5 and 0.5 respectively.

[0138] In the present application, the second loss function value of the generator can also be determined based on each of the image labels and the predicted labels corresponding to each of the encrypted images by using the following formula:

[0139]

[0140] wherein, the execution order of S205 and S206 is not limited, S205 and S206 can be executed simultaneously, or S205 can be executed before S206, or S206 can be executed before S205.

[0141] S207, if the first loss function value and the second loss function value are both less than a preset loss threshold or the current iteration number reaches a second preset iteration number, it is determined that the generative adversarial network training is completed, and the generator is determined as the encryption region selection model; otherwise, the network parameters of the generator and the network parameters of the discriminator are adjusted, and the execution of inputting the sample image, the privacy level of the sample image and the response time tolerance corresponding to the sample image into the generative adversarial network for each sample image is returned.

[0142] In the present application, the second preset iteration number can be set to 1000 or 2000, etc. The preset loss threshold can be set to 0.05 or 0.04, etc.

[0143] In the present application, the generator and the discriminator of the generative adversarial network can be trained together. The training target of the discriminator is to accurately identify the content of the sample image. The training target of the generator is to generate an encryption region selection matrix, so that the encrypted image cannot be accurately identified by the discriminator.

[0144] When the generative adversarial network training is completed, the second device can determine the generator as the encryption region selection model, and distribute the encryption region selection model to the first device, so that the first device can use the encryption region selection model to encrypt the to-be-processed task, not only can improve the encryption efficiency and accuracy of the first device, but also can determine the encryption overhead through the encryption region selection model, so that the allocation decision of the first device for the to-be-processed task is more accurate.

[0145] Step A3, determining a plurality of to-be-encrypted pixel points according to the pixel values of the pixel points in the encryption region.

[0146] In the present application, among the pixel points in the encryption region that need to be encrypted, the pixel points with pixel values greater than a preset pixel value are determined as to-be-encrypted pixel points. The preset pixel value can be set to 250 or 251. The number of to-be-encrypted pixel points is denoted as k, and the horizontal coordinates of the coordinate points of the to-be-encrypted pixel points are stored in the array Rindex, and the vertical coordinates are stored in the array Cindex.

[0147] Step A4, determining the predicted encryption time delay and the predicted encryption energy consumption corresponding to the plurality of to-be-encrypted pixel points as the predicted encryption overhead corresponding to each to-be-processed task.

[0148] In the present application, the predicted encryption time delay may include the delay of generating the encryption region selection matrix , the first encryption delay of encrypting the to-be-processed task based on the encryption region selection matrix , and the second encryption delay :

[0149]

[0150] In the present application, the following formula can be used to calculate the delay of generating the encryption region selection matrix :

[0151]

[0152] wherein, is the total number of image pixels included in the task to be processed, is the average CPU cycle required for the encryption region selection to make one forward inference on a single pixel, is the local CPU frequency of the first device.

[0153] In the present application, the following formula can be used to calculate the first encryption delay :

[0154]

[0155] wherein, NP is the number of pre-iterations, is the number of pixels to be encrypted, is the average cycle required for one iteration.

[0156] In the present application, the following formula can be used to calculate the second delay :

[0157]

[0158] wherein, is the cycle of one XOR processing.

[0159] In the present application, the encryption region ratio can be used as the predicted encryption energy consumption. The predicted encryption delay and the predicted encryption energy consumption are used together as the predicted encryption overhead.

[0160] In one possible implementation, the determining, according to the first computing performance information, of the first predicted computing overhead corresponding to the execution of each of the tasks to be processed by the first device can include steps B1-B2:

[0161] Step B1, determining, according to the first computing performance information, the first predicted delay and the first predicted energy consumption corresponding to the execution of each of the tasks to be processed by the first device.

[0162] In the present application, the following formula can be used to determine, according to the first computing performance information, the first predicted delay corresponding to the execution of each of the tasks to be processed by the first device:

[0163]

[0164] in, Execute the i-th pending task locally on the first device. The corresponding first prediction delay, This refers to the local CPU frequency of the first device. For the i-th pending task Required resource parameters This represents the local device.

[0165] In this application, the following formula can be used to determine the first predicted energy consumption corresponding to each of the tasks to be processed when executed by the first device, based on the first computing performance information:

[0166]

[0167] in, Execute the i-th pending task locally on the first device. The first predicted energy consumption corresponding to the time, The device power of the first device in each CPU cycle. δ and γ are chip-related constants of the first device.

[0168] Step B2: Based on the first preset weight information, the first prediction delay and the first prediction energy consumption are weighted and summed to obtain the first prediction calculation cost.

[0169] In this application, the first preset weight information refers to the weights assigned to the first prediction delay and the first prediction energy consumption based on their importance to task completion. For example, if the first prediction delay and the first prediction energy consumption have the same importance to task completion, then the same weight can be assigned to them. The first prediction delay and the first prediction energy consumption are then weighted and summed to obtain the first prediction computational cost.

[0170] In one possible implementation, the link information includes the link bandwidth, link channel gain, and noise power of the target transmission link. Determining the second prediction computational overhead corresponding to each of the tasks to be processed through the second device based on the second computational performance information and the link information may include steps C1-C2:

[0171] Step C1: Based on the second calculation performance information, the link bandwidth, the link channel gain, and the noise power, determine the second predicted latency and the second predicted power consumption corresponding to each of the tasks to be processed when executed by the second device.

[0172] In this application, if it is necessary to offload the tasks to be processed to the second device and have the second device execute each of the tasks to be processed, the processing latency and processing power consumption of the second device, as well as the link transmission latency and link transmission power consumption during the process of offloading the tasks to be processed to the second device, need to be considered.

[0173] The second predicted latency for each of the tasks to be processed executed through the second device can be determined using the following formula:

[0174]

[0175] in, Execute the i-th pending task for the second device. The corresponding second prediction delay, Execute the i-th pending task for the second device. The corresponding processing delay To process the i-th task Link transmission delay during the unloading process to the second device. , 'r' represents the CPU frequency of the second device.

[0176] The following formula can be used to determine it:

[0177]

[0178] in, This represents the allocation decision that assigns the task to be processed to the second device for execution. Characterizes the uplink rate of the target transmission link. h t N0 is the uplink channel gain, W is the noise power, and P is the link bandwidth. t This refers to the internal link transmission power.

[0179] The second predicted energy consumption for each of the tasks to be processed can be determined using the following formula:

[0180]

[0181] in, Execute the i-th pending task for the second device. The second predicted energy consumption corresponding to the time, Execute the i-th pending task for the second device. The corresponding processing energy consumption, To process the i-th task Link transmission energy consumption during the unloading process to the second device. , The device power of the second device.

[0182] The following formula can be used to determine:

[0183]

[0184] Step C2, the second prediction calculation overhead is obtained by weighted sum of the second prediction time delay and the second prediction energy consumption based on the second preset weight information.

[0185] In the present application, the second preset weight information refers to the weight set for the second prediction time delay and the second prediction energy consumption according to the importance of the task completion processing. For example, if the importance of the time delay to the task completion processing is higher than the energy consumption, the weight of the second prediction time delay can be set to be greater than the weight of the second prediction energy consumption. If the importance of the time delay to the task completion processing is the same as the energy consumption, the weight of the second prediction time delay can be set to be the same as the weight of the second prediction energy consumption. Then the second prediction time delay and the second prediction energy consumption are weighted and summed to obtain the second prediction calculation overhead.

[0186] In one possible implementation, Figure 3 A training process schematic diagram of the decision model provided by the embodiments of the present application is shown, as shown in Figure 3 The training process of the decision model includes:

[0187] S301, for the online network and the target network included in the double-network reinforcement learning model to be trained, the network parameters of the online network are synchronized to the target network.

[0188] The network structures of the online network and the target network are the same. Since the single network is prone to drift and non-convergence problems in the updating process, in order to avoid the shock effect caused by state changes, the double-network reinforcement learning model is adopted in the present application. The allocation decision determined by the decision model trained by the double-network reinforcement learning model to be trained is more stable and more accurate.

[0189] In the present application, the network structure of the online network and the target network includes an input layer, a hidden layer and an output layer. Taking the online network as an example, the input layer is used to receive the state vector The dimension of the input layer is equal to the sum of the components of the state vector The hidden layer is used to extract the features of the state vector The hidden layer can include 2 to 4 layers of structure, and each layer can include 128 to 512 neurons. The neurons of the hidden layer can be activated by using the activation function ReLU to improve the nonlinear feature extraction capability of the neurons. The output layer is used to output the length of the decision number a vector. The network parameters of the online network can be θ, and the network parameters of the target network are Before training the double-network reinforcement learning model to be trained, the network parameters of the online network can be synchronized to the target network, that is, the parameters of the target network are copied as θ. After initializing the online network and the target network of the double-network reinforcement learning model to be trained, the network parameters of the online network can also be synchronized to the target network to maintain the stability of the double network.

[0190] S302, sample information of a preset number of interaction samples is obtained based on an experience replay pool, the sample information of the interaction samples includes device state information of a sample device, an allocation decision, a standard reward, and device state information after the allocation decision is executed, and the experience replay pool is used to store sample information of a plurality of interaction samples.

[0191] In this application, a plurality of interaction samples and sample information of the interaction samples are stored in the experience replay pool. The experience replay pool can break the time series data correlation and improve sample utilization. The experience replay pool has a storage upper limit. When the interaction samples and the sample information of the interaction samples stored in the experience replay pool reach the storage upper limit, if there are new interaction samples, the interaction samples stored in the experience replay pool for a long time can be deleted, and the new interaction samples and the sample information of the interaction samples can be stored in the experience replay pool to maintain the generality of the samples in the experience replay pool. The storage upper limit can be set according to application requirements, which is not limited here.

[0192] The preset number is equal to the number of interaction samples that can be batch processed by the double-network reinforcement learning model to be trained. For example, if the number of batch training is set to 10, the preset number can also be set to 10.

[0193] The sample information of the interaction samples includes device state information of a sample device, an allocation decision, a standard reward, and device state information after the allocation decision is executed, and the experience replay pool is used to store sample information of a plurality of interaction samples. The sample device can be a first device. The device state information refers to the current state of the first device, and the standard reward refers to the decision value of the allocation decision. For example, at time t, there are N tasks to be processed in the first device: The tolerance response time of the N tasks to be processed is The privacy level of the N tasks to be processed is The value of the privacy level may be In this application, the encryption region selection model can be used to determine the encryption region corresponding to the N tasks to be processed, and the encryption region is used to determine the encryption overhead In the current state s , , the state vector of the current state can be obtained, which characterizes the predicted encryption overhead of the to-be-processed task, the current device state information, the first predicted computation overhead, the second predicted computation overhead t represents the network status of the first device at time t, e t represents the percentage of the remaining power of the first device at time t, p t represents the CPU current load occupancy rate of the first device at time t, and represent the delay and energy consumption of executing N to-be-processed tasks locally on the first device, and represent the total delay and total energy consumption consumed by offloading N to-be-processed tasks to the second device, and MASK represents an encryption region selection matrix. In the current state s t , if the allocation decision a t = 0, that is, the N to-be-processed tasks need to be processed locally on the first device, if a t = 1, that is, the N to-be-processed tasks need to be processed on the second device. After the selection decision a t is selected in the current state s t , the state of the first device will become s t+1 , and a corresponding reward R t is obtained. In order to minimize the delay and energy consumption and the power e t of the first device and other information, the reward R t can be defined. When a t = 0, that is, the delay overhead and the energy consumption overhead of the first device for locally processing the N to-be-processed tasks can be defined as follows:

[0194] ;

[0195]

[0196] The total overhead, that is, the first predicted computation overhead is the weighted average of the delay overhead and the energy consumption overhead : . Wherein, and are weights corresponding to the delay overhead and the energy consumption overhead , respectively.

[0197] When a t = 1, that is, the delay overhead of offloading the N to-be-processed tasks to the second device for processing by the second device can be defined as follows:and energy consumption overheads :

[0198] ;

[0199]

[0200] total overheads, i.e., second prediction computation overheads latency overheads and energy consumption overheads weighted average of: wherein, and latency overheads and energy consumption overheads corresponding weights, respectively.

[0201] based on the first prediction computation overheads of N to-be-processed tasks when executed locally on the first device and the second prediction computation overheads when executed on the second device a reward function of the first device taking a decision in state .

[0202] wherein, and represent latency and energy consumption of executing the to-be-processed task locally on the first device, and represent total latency and total energy consumption consumed by offloading the to-be-processed task to the second device, MASK represents an encryption region selection matrix corresponding to the to-be-processed task, is the encryption latency corresponding to the encryption region of the to-be-processed task, represents a cycle of one XOR processing, is the allocation decision, P(MASK) represents the proportion of the encryption region in the to-be-processed task, , and are corresponding weights of , and , respectively, and are weights corresponding to and P(MASK).

[0203] S303, input sample information of a preset number of interaction samples into the target network to obtain a first prediction reward after the target network executes an allocation decision for each interaction sample.

[0204] In the present application, the value of the decision output by the target network can be used as the first predicted reward. For example, the value of the decision output by the target network based on the network parameters of the target network is . .

[0205]

[0206] wherein, is a constant for the network parameters , is the maximum value estimate for all possible actions in the next state, is the reward corresponding to the decision generated by the target network.

[0207] S304, sample information of a preset number of interaction samples is input into the online network, and a second predicted reward of the online network after executing the allocation decision for each interaction sample is obtained.

[0208] In the present application, the input layer of the online network is used to receive the state vector , the hidden layer is used to extract the features of the state vector , and the output layer is used to output a vector with a length of the number of decisions . The value of the decision output by the online network can be obtained based on the decision output by the output layer. For example, the value of the decision output by the online network based on the network parameters of the online network is . The value of the decision can be used as the second predicted reward.

[0209] In one possible implementation, the second predicted reward corresponding to each interaction sample can also be determined by the following formula:

[0210]

[0211] wherein, is the device state information of the sample device, , n t2 represents the network condition of the sample device at time t, t2 represents the percentage of the remaining power of the sample device at time t, represents the current CPU load occupancy rate of the sample device at time t, and represent the delay and energy consumption of the sample device in executing the to-be-processed task locally, and ​This represents the total latency and total energy consumption consumed in offloading the task to be processed to the second device. MASK represents the encryption region selection matrix corresponding to the task to be processed. The encryption latency corresponding to the encrypted area of ​​the task to be processed. express The cycle of one XOR operation, For allocation decisions, To execute allocation decisions The second predicted reward, The computational overhead of executing the task to be processed locally on the sample device. P2(MASK) represents the computational overhead of offloading the task to be processed from the sample device to the second device and having the second device execute the task. P2(MASK) represents the proportion of the encrypted area in the task to be processed. , and They are respectively , and The corresponding weights and for The weights corresponding to P2(MASK). In this application, the sample device can be the first device.

[0212] In one possible implementation, the step of inputting sample information of a preset number of interaction samples into the online network to obtain a second predicted reward after the online network performs an allocation decision for each of the interaction samples may include steps D1-D3:

[0213] Step D1: Input the sample information of a preset number of interactive samples into the online network, and the online network executes the command.

[0214] Step D2: Determine the allocation decision for each of the interaction samples based on the preset training hyperparameters.

[0215] In this application, the preset training hyperparameters may include a preset exploration rate ε; the step of determining the allocation decision for each interaction sample based on the preset training hyperparameters includes: for each interaction sample, generating a random number based on the device status information of the sample device corresponding to the interaction sample; and in response to the random number being not less than the preset exploration rate, determining the allocation decision for the interaction sample based on the target probability and a greedy model.

[0216] Setting a preset exploration rate ε can tilt the online network towards decisions with higher value. For example, each time based on the state vector... When making a choice, the online network can first generate a random number in the interval [0,1]. If the random number is less than... Then, based on the principle of uniform exploration, a random allocation decision is selected. ; if the random number is not less than , the probability of According to the greedy model, the allocation decision for the interaction sample is determined. Specifically, the allocation decision of the interaction sample can be determined by the following formula when the random number is not less than

[0217]

[0218] In this application, with the increase of the number of training iterations, the , so that the online network can concentrate on using the learned strategy training parameters.

[0219] In this application, the preset training hyperparameters can also include a discount factor γ, a learning rate η, a training batch size, and a network parameter synchronization interval N, etc. γ ∈ [0, 1].

[0220] Step D3, executing the allocation decision for each of the interaction samples to obtain a second predicted reward corresponding to each of the interaction samples.

[0221] S305, determining a loss function of the to-be-trained double-network reinforcement learning model based on the first predicted reward and the second predicted reward.

[0222] In this application, the loss function of the to-be-trained double-network reinforcement learning model can be calculated by the following formula

[0223]

[0224] Wherein, B is the training batch size.

[0225] S306, determining a target evaluation index based on the loss function.

[0226] S307, if the target evaluation index does not meet the training completion condition, adjusting the network parameters of the online network.

[0227] The first preset number of iterations can be set to 5 or 10, etc. The network parameters of the online network are synchronized to the target network every first preset number of iterations. This can make the training converge more stably and gently, and also reduce the overestimation bias. The network parameters of the online network can be adjusted by the Adam optimizer or the SGD optimizer.

[0228] S308, if the target evaluation index meets the training completion condition, determining that the training is completed.

[0229] ​​In the present application, the target evaluation index includes a value of the loss function or a gradient value of the loss function; and the training completion condition includes that the value of the loss function is less than a preset loss function threshold or the gradient value of the loss function is less than a preset gradient threshold.

[0230] In the present application, the gradient of the loss function can be determined by the following formula

[0231]

[0232] The gradient may be a constant, and the gradient acts on .

[0233] In the present application, if the value of the loss function is less than a preset loss function threshold or the gradient value of the loss function is less than a preset gradient threshold, it can be determined that the training is completed. The preset loss function threshold can be set to 0.05 or 0.04, etc. The preset gradient threshold can be set to 1.0 or 2.0, etc. In the present application, the target evaluation index can also include an average reward and a decision effect, and the decision effect can be an index such as a task completion rate, a task energy consumption, and a task time delay. The training completion condition can include that the average reward is greater than a preset reward threshold or the decision effect reaches a preset effect. The preset effect can be that the task completion rate is greater than a preset completion rate, the task energy consumption is less than a preset energy consumption, or the task time delay is less than a preset time delay. The preset completion rate can be set to 0.98 or 0.99, and the preset energy consumption and the preset time delay can be set according to the first device state, which is not limited here.

[0234] S309, in the case that the current iteration number reaches a first preset iteration number or the training is completed, synchronizing network parameters of the online network to the target network, and determining the trained double-network reinforcement learning model after updating the parameters as a decision model.

[0235] In the present application, in the process of training the trained double-network reinforcement learning model, the features of the encryption region are introduced, such as the encryption overhead and the proportion of the encryption region, etc. Through the features of the encryption region, the trained reinforcement learning model can be guided to seek the optimal balance between the encryption overhead and the computing performance under the premise of ensuring privacy, so that the decision model obtained by training can directly perceive and respond to the cost brought by the privacy protection operation. Thus, the decision model can determine more accurate allocation decisions by referring to more comprehensive information.

[0236] In one possible implementation manner, Figure 4 A task encryption process schematic diagram provided by an embodiment of the present application is shown, as Figure 4 ​In a case where the target allocation decision is that each of the second devices performs each of the to-be-processed tasks, the computing task allocation method further includes:

[0237] S401, generating an encryption key based on the device identifier of the first device and a random number set maintained by the first device.

[0238] In this application, the periodically updated random numbers maintained by the second device can constitute the random number set. Three random numbers can be selected from the random number set, and the three random numbers are subjected to bitwise XOR operation with the device identifier of the second device to obtain three intermediate values corresponding to the three random numbers respectively. The device identifier can be a device ID.

[0239] The three intermediate values corresponding to the three random numbers are subjected to hash processing to obtain three hash values. For example, SHA-256 (hash algorithm) can be used to hash process the intermediate values of the three random numbers to obtain three encryption parameters: control parameter u of Logistic mapping, initial value x0, and pre-iteration number NP.

[0240] Each first device has a set of encryption parameters, and each first device cannot obtain the encryption parameters of other first devices. Each second device also has a set of encryption parameters, and each second device cannot obtain the encryption parameters of other second devices.

[0241] The first device can use the encryption parameters (u, x0, NP) as the encryption key.

[0242] S402, generating a first encryption sequence and a second encryption sequence according to the encryption key and the chaotic encryption algorithm, the first encryption sequence and the second encryption sequence including the same number of parameters.

[0243] The first device can generate the first encryption sequence and the second encryption sequence according to the chaotic encryption algorithm, according to Iterating the encryption parameters for NP+2k times, and retaining the last 2k iteration results to constitute a chaotic sequence Taking the first k iteration values of the chaotic sequence to constitute the first encryption sequence, and denoting the first encryption sequence as X1, Taking the last k iteration values of the chaotic sequence to constitute the second encryption sequence, and denoting the first encryption sequence as X2, Wherein, the value of k can be the number of to-be-encrypted pixel points.

[0244] S403, performing permutation processing on the plurality of to-be-encrypted pixel points based on the first encryption sequence to obtain an intermediate ciphertext sequence.

[0245] The plurality of to-be-encrypted pixel points are subjected to the shuffling processing by the first encryption sequence, the positions of the to-be-encrypted pixel points can be shuffled, the adjacent pixel points in the plaintext no longer appear in the adjacent positions in the ciphertext, thereby hiding the spatial structure of the image and improving the reliability of the encryption. Specifically, the first encryption sequence X1 can be sorted in ascending order to obtain a sorted sequence X1', and an array T is used to record the positions of the elements in the first encryption sequence X1'. Specifically, the array T[1…k] can record the position index of each element in the first encryption sequence X1' in the sorted sequence X1'. For example, if the first element (i.e. the minimum value element) in the sorted sequence X1' is the 7th element in the first encryption sequence X1, the array T can record the position index of the first element in the sorted sequence X1' in the first encryption sequence X1: T[1]=7; if the second element (i.e. the minimum value element) in the sorted sequence X1' is the 10th element in the first encryption sequence X1, the array T can record the position index of the first element in the sorted sequence X1' in the first encryption sequence X1: T[2]=10.

[0246] Specifically, for the i-th to-be-encrypted pixel point (Rindez(i), Cindex(i)) in the k to-be-encrypted pixel points, the ciphertext value of the to-be-encrypted pixel point can be replaced by the value of the pixel point at the T(i)-th element position in the plaintext:

[0247] C'(Rindez(i),Cindex(i))=P(Rindez(T(i)),Cindez(T(i)))(i=1,2,…,k)

[0248] Wherein, C' is the intermediate ciphertext after the shuffling processing, P is the original plaintext image, and (Rindex(i), Cindex(i)) is the horizontal coordinate and vertical coordinate of the i-th pixel to be encrypted in the image.

[0249] The element value in the second encryption sequence X2 can be converted into an integer between 0 and 255 by using the following formula: mod is the integer function.

[0250] Further, each element in the second encryption sequence X2 can be processed by using the following formula: so that each element in the sequence falls within the interval:

[0251]

[0252] ​​​​For the i-th pixel point (Rindez(i), Cindex(i)) in the k pixel points to be encrypted, the following scrambling operation can be used to obtain the intermediate ciphertext sequence C':

[0253] , i = 1, 2, …, k

[0254] S404, based on the second encryption sequence and the intermediate ciphertext sequence, performing diffusion processing on the plurality of pixel points to be encrypted to obtain an encrypted image of each of the tasks to be processed.

[0255] Specifically, the intermediate ciphertext sequence C' can be processed by the following formula to obtain the encrypted image of the task to be processed :

[0256] When i = 1,

[0257] When 1 < i <= k,

[0258] wherein, represents the XOR operation.

[0259]

[0260] When the first device unloads the encrypted task to be processed to the second device, the second device can decrypt the encrypted region in the task to be processed according to the inverse process of the encryption process, and restore the ciphertext C to the plaintext image P.

[0261] In the present application, the task to be processed is encrypted by a chaotic encryption algorithm, which combines pixel position scrambling and pixel value diffusion based on a chaotic sequence, can improve the security and attack resistance of encryption, while maintaining a low computational complexity and improving the task processing efficiency.

[0262] Based on the same inventive concept, according to the computing task allocation method provided by the above embodiments of the present application, correspondingly, another embodiment of the present application further provides a computing task allocation device, a structure diagram of which is shown as Figure 5 specifically comprising:

[0263] The first information acquisition module 501 is configured to acquire at least one task to be processed of the first device, current device state information of the first device, and first computing performance information of the first device.

[0264] The second information acquisition module 502 is configured to acquire second computing performance information of the second device and link information of a target transmission link, the target transmission link being a transmission link between the first device and the second device.

[0265] The overhead determination module 503 is configured to determine a predicted encryption overhead corresponding to each of the to-be-processed tasks; determine a first predicted computation overhead corresponding to each of the to-be-processed tasks when the first device is used to execute the to-be-processed tasks according to the first computation performance information; and determine a second predicted computation overhead corresponding to each of the to-be-processed tasks when the second device is used to execute the to-be-processed tasks according to the second computation performance information and the link information.

[0266] The decision module 504 is configured to determine a target allocation decision for each of the to-be-processed tasks based on the predicted encryption overhead, the current device state information, the first predicted computation overhead, the second predicted computation overhead, and a pre-trained decision model, wherein the decision model is a double-network reinforcement learning model.

[0267] The computing task allocation apparatus provided in the embodiments of the present application associates the encryption overhead of the to-be-processed tasks with the allocation decision, considers the computation amount and the transmission overhead caused by the encryption operation when determining the computation overhead corresponding to the to-be-processed tasks, takes the privacy protection cost of the to-be-processed tasks into account, and makes the determined target allocation decision more accurate. In addition, the double-network structure of the double-network reinforcement learning model can make the decision process more stable and have better convergence. The pre-trained decision model of the double-network reinforcement learning model is used to make the task allocation decision in the present application, which can comprehensively consider the computation overhead of the to-be-processed tasks, balance the computation performance of the devices, the privacy of the to-be-processed tasks, the computation cost and the encryption cost, and make the generated target allocation decision be able to realize the collaborative optimization of the execution efficiency of the computing tasks and the security of the task data.

[0268] In an implementation, the device further comprises a model training module (not shown in the figure) configured to: synchronize network parameters of an online network to a target network for the online network and the target network included in a double-network reinforcement learning model to be trained, the online network and the target network having the same network structure; obtain sample information of a preset number of interaction samples based on an experience replay pool, the sample information of the interaction samples including device state information of a sample device, an allocation decision, a standard reward, and device state information after the allocation decision is executed, the experience replay pool being configured to store sample information of a plurality of interaction samples; input the sample information of the preset number of interaction samples into the target network to obtain a first predicted reward of the target network after the allocation decision is executed for each of the interaction samples; input the sample information of the preset number of interaction samples into the online network to obtain a second predicted reward of the online network after the allocation decision is executed for each of the interaction samples; determine a loss function of the double-network reinforcement learning model to be trained based on the first predicted reward and the second predicted reward; determine a target evaluation index based on the loss function; if the target evaluation index does not reach a training completion condition, adjust the network parameters of the online network; if the target evaluation index reaches the training completion condition, determine that the training is completed; and in a case where a current iteration number reaches a first preset iteration number or the training is completed, synchronize the network parameters of the online network to the target network, and determine the double-network reinforcement learning model after the network parameters are updated as a decision model.

[0269] In an implementation, the target evaluation index includes a value of the loss function or a gradient value of the loss function, and the training completion condition includes that the value of the loss function is less than a preset loss function threshold or the gradient value of the loss function is less than a preset gradient threshold.

[0270] In an implementation, the model training module is specifically configured to input the sample information of the preset number of interaction samples into the online network, and the online network is configured to: determine the allocation decision for each of the interaction samples based on a preset training hyperparameter; and execute the allocation decision for each of the interaction samples to obtain the second predicted reward corresponding to each of the interaction samples.

[0271] In an implementation, the model training module is specifically configured to determine the second predicted reward corresponding to each of the interaction samples by using the following formula:

[0272]

[0273] wherein, is the device state information of the sample device, , n t2 represents a network condition of the sample device at time t, e t2This represents the percentage of remaining battery power of the sample device at time t. This represents the current CPU load utilization rate of the sample device at time t. and This indicates the latency and energy consumption when the task to be processed is executed locally on the sample device. and This represents the total latency and total energy consumption consumed in offloading the task to be processed to the second device. MASK represents the encryption region selection matrix corresponding to the task to be processed. The encryption latency corresponding to the encrypted area of ​​the task to be processed. express The cycle of one XOR operation, For allocation decisions, To execute allocation decisions The second predicted reward, The computational overhead of executing the task to be processed locally on the sample device. P2(MASK) represents the computational overhead of offloading the task to be processed from the sample device to the second device and having the second device execute the task. P2(MASK) represents the proportion of the encrypted area in the task to be processed. , and They are respectively , and The corresponding weights and for The weights corresponding to P2(MASK).

[0274] In one possible implementation, the preset training hyperparameters include a preset exploration rate;

[0275] The model training module is specifically used to generate random numbers for each interaction sample based on the device status information of the sample device corresponding to the interaction sample; and in response to the random number being not less than the preset exploration rate, to determine the allocation decision for the interaction sample based on the target probability and the greedy model.

[0276] In an implementation, the overhead determination module 503 is specifically configured to obtain at least one task image of each of the to-be-processed tasks, an image label of each of the task images, a privacy level of each of the task images, and a response time tolerance of each of the task images; input the task image, the image label of the task image, the privacy level of the task image, and the response time tolerance of the task image into an encryption region selection model to obtain an encryption region in the task image, the encryption region selection model being obtained by pre-training of the second device based on a generative adversarial network; determine a plurality of to-be-encrypted pixel points according to pixel values of each of the pixel points in the encryption region; and determine a predicted encryption time delay and a predicted encryption energy consumption corresponding to the plurality of to-be-encrypted pixel points as the predicted encryption overhead corresponding to each of the to-be-processed tasks.

[0277] In an implementation, the generative adversarial network includes a generator and a discriminator, and a training process of the encryption region selection model includes: obtaining a sample labeled image dataset, the sample labeled image dataset including a plurality of sample images, an image label of each of the sample images, a privacy level of each of the sample images, and a response time tolerance corresponding to each of the sample images; inputting, for each of the sample images, the sample image, the privacy level of the sample image, and the response time tolerance corresponding to the sample image into the generative adversarial network, the generator of the generative adversarial network generating an encryption region corresponding to the sample image based on the sample image, the privacy level, and the response time tolerance; performing encryption processing on each of the sample images based on the encryption region of each of the sample images to obtain an encrypted image corresponding to each of the sample images; inputting each of the sample images and each of the encrypted images into the generative adversarial network, the discriminator of the generative adversarial network generating a predicted label corresponding to each of the sample images and each of the encrypted images, respectively; determining a first loss function value of the discriminator based on the predicted label corresponding to each of the sample images and the image label and the predicted label corresponding to each of the encrypted images; determining a second loss function value of the generator based on each of the image labels and the predicted label corresponding to each of the encrypted images; if the first loss function value and the second loss function value are both less than a preset loss threshold or a current iteration number reaches a second preset iteration number, determining that the training of the generative adversarial network is completed, and determining the generator as the encryption region selection model; otherwise, adjusting network parameters of the generator and network parameters of the discriminator, and returning to perform, for each of the sample images, inputting the sample image, the privacy level of the sample image, and the response time tolerance corresponding to the sample image into the generative adversarial network.

[0278] In an implementation, the link information includes a link bandwidth, a link channel gain, and a noise power of the target transmission link.

[0279] The overhead determination module 503 is configured to determine, according to the second computing performance information, the link bandwidth, the link channel gain and the noise power, a second predicted time delay and a second predicted energy consumption corresponding to execution of each of the to-be-processed tasks by the second device; and perform weighted summation on the second predicted time delay and the second predicted energy consumption based on second preset weight information to obtain a second predicted computing overhead.

[0280] In an implementation, the decision module 504 is further configured to generate an encryption key based on a device identifier of the first device and a set of random numbers maintained by the first device; generate a first encryption sequence and a second encryption sequence according to the encryption key and a chaotic encryption algorithm, the first encryption sequence and the second encryption sequence including the same number of parameters; perform a permutation process on the plurality of to-be-encrypted pixel points based on the first encryption sequence to obtain an intermediate ciphertext sequence; and perform an expansion process on the plurality of to-be-encrypted pixel points based on the second encryption sequence and the intermediate ciphertext sequence to obtain the encrypted image of each of the to-be-processed tasks.

[0281] In an implementation, the target allocation decision includes a decision of executing each of the to-be-processed tasks by the first device, and a decision of offloading each of the to-be-processed tasks to the second device and executing each of the to-be-processed tasks by the second device.

[0282] The application further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the computing task allocation method when executing the program.

[0283] The application further provides a storage medium containing computer executable instructions, wherein the computer executable instructions are used to execute the computing task allocation method when executed by a computer processor.

[0284] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the application described and / or claimed in this document.

[0285] As Figure 6As shown, the electronic device 600 includes a computing unit 601 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. Various programs and data required for the operation of the electronic device 600 can also be stored in the RAM 603. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0286] A plurality of components in the electronic device 600 are connected to the I / O interface 605, including an input unit 606 such as a keyboard, a mouse, and the like; an output unit 607 such as various types of displays, a speaker, and the like; the storage unit 608 such as a magnetic disk, an optical disk, and the like; and a communication unit 609 such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 609 allows the electronic device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0287] The computing unit 601 can be various general and / or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, and the like. The computing unit 601 performs various methods and processes described above, such as the computing task allocation method. For example, in some embodiments, the computing task allocation method can be implemented as a computer software program that is tangibly embodied in a machine-readable medium, such as the storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the computing task allocation method described above can be performed. Alternatively, in other embodiments, the computing unit 601 can be configured to perform the computing task allocation method by any other appropriate means, such as by means of firmware.

[0288] The various embodiments of the systems and techniques described above can be implemented in digital electronic circuitry, integrated circuitry, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0289] Program code for carrying out methods of the present application can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces a means for implementing the functions / acts specified in the flowcharts and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0290] In the context of the present application, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical conductors, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0291] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0292] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0293] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server is generally established by computer programs running on the respective computers and having a client-server relationship to each other. The servers can be cloud servers, servers of a distributed system, or servers combined with a blockchain.

[0294] It should be understood that various forms of flow shown above can be used, with steps re-ordered, added, or removed. For example, the steps recited in the present disclosure can be performed in parallel, in series, or in a different order, as long as the desired results of the technology disclosed herein are achieved, and the present disclosure is not limited herein.

[0295] In addition, the terms "first", "second", etc., are used herein only to describe different instances, and do not imply or suggest relative importance or imply the number of indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.

[0296] The above merely provides the specific implementation of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of the changes or replacements within the technical range disclosed by the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method of allocating computing tasks, characterized by, The method comprises: obtaining at least one to-be-processed task of a first device, current device state information of the first device, and first computing performance information of the first device; obtaining second computing performance information of a second device and link information of a target transmission link, the target transmission link being a transmission link between the first device and the second device; determining a predicted encryption overhead corresponding to each of the to-be-processed tasks; determining, according to the first computing performance information, a first predicted computing overhead corresponding to execution of each of the to-be-processed tasks by the first device, and determining, according to the second computing performance information and the link information, a second predicted computing overhead corresponding to execution of each of the to-be-processed tasks by the second device; determining, based on the predicted encryption overhead, the current device state information, the first predicted computing overhead, the second predicted computing overhead, and a pre-trained decision model, a target allocation decision for each of the to-be-processed tasks, the decision model being a double-network reinforcement learning model; wherein the determining of the predicted encryption overhead corresponding to each of the to-be-processed tasks comprises: obtaining at least one task image of each of the to-be-processed tasks, an image label of each of the task images, a privacy level of each of the task images, and a response time tolerance of each of the task images; inputting each of the task images, the image label of each of the task images, the privacy level of each of the task images, and the response time tolerance of each of the task images into an encryption region selection model to obtain an encryption region in each of the task images, the encryption region selection model being pre-trained by the second device based on a generative adversarial network; determining a plurality of to-be-encrypted pixel points according to pixel values of each of the pixel points in the encryption region; determining a predicted encryption time delay and a predicted encryption energy consumption corresponding to the plurality of to-be-encrypted pixel points as the predicted encryption overhead corresponding to each of the to-be-processed tasks.

2. The method of claim 1, wherein, The training process of the decision model comprises: synchronizing network parameters of an online network included in a to-be-trained double-network reinforcement learning model to a target network, the online network and the target network having the same network structure; obtaining sample information of a preset number of interaction samples based on an experience replay pool, the sample information of the interaction samples including device state information of a sample device, an allocation decision, a standard reward, and device state information after execution of the allocation decision, the experience replay pool being configured to store sample information of a plurality of interaction samples; inputting the sample information of the preset number of interaction samples into the target network to obtain a first predicted reward of the target network after execution of the allocation decision for each of the interaction samples; inputting the sample information of the preset number of interaction samples into the online network to obtain a second predicted reward of the online network after execution of the allocation decision for each of the interaction samples; determining a loss function of the to-be-trained double-network reinforcement learning model based on the first predicted reward and the second predicted reward; determining a target evaluation index based on the loss function. If the target evaluation index does not reach the training completion condition, adjusting the network parameters of the online network; If the target evaluation index reaches the training completion condition, determining that the training is completed; In the case that the current iteration number reaches a first preset iteration number or the training is completed, synchronizing the network parameters of the online network to the target network, and determining the trained double-network reinforcement learning model after updating the parameters as a decision model.

3. The method of claim 2, wherein, The target evaluation index includes a value of the loss function or a gradient value of the loss function; and the training completion condition includes that the value of the loss function is less than a preset loss function threshold or the gradient value of the loss function is less than a preset gradient threshold.

4. The method of claim 2, wherein, The inputting of the sample information of the preset number of interaction samples into the online network to obtain the second predicted reward of the online network after performing the allocation decision for each interaction sample includes: The inputting of the sample information of the preset number of interaction samples into the online network includes that the online network performs: determining the allocation decision for each interaction sample based on a preset training hyperparameter; performing the allocation decision for each interaction sample to obtain the second predicted reward corresponding to each interaction sample.

5. The method of claim 4, wherein, The performing of the allocation decision for each interaction sample to obtain the second predicted reward corresponding to each interaction sample includes: determining the second predicted reward corresponding to each interaction sample by using the following formula: in, This provides the device status information for the sample equipment. n t2 e represents the network status of the sample device at time t. t2 This represents the percentage of remaining battery power of the sample device at time t. This represents the current CPU load utilization rate of the sample device at time t. and This indicates the latency and energy consumption when the task to be processed is executed locally on the sample device. and This represents the total latency and total energy consumption consumed in offloading the task to be processed to the second device. MASK represents the encryption region selection matrix corresponding to the task to be processed. The encryption latency corresponding to the encrypted area of ​​the task to be processed. express The cycle of one XOR operation, For allocation decisions, To execute allocation decisions The second predicted reward, The computational overhead of executing the task to be processed locally on the sample device. P2(MASK) represents the computational overhead of offloading the task to be processed from the sample device to the second device and having the second device execute the task. P2(MASK) represents the proportion of the encrypted area in the task to be processed. , and They are respectively , and The corresponding weights and for The weights corresponding to P2(MASK).

6. The method of claim 4, wherein, The preset training hyperparameter includes a preset exploration rate. The determining of the allocation decision for each interaction sample based on the preset training hyperparameter includes: for each interaction sample, generating a random number based on the device state information of the sample device corresponding to the interaction sample; in response to the random number being not less than the preset exploration rate, determining the allocation decision for the interaction sample based on a target probability and a greedy model.

7. The method of claim 1, wherein, The generative adversarial network includes a generator and a discriminator, and the training process of the encryption region selection model includes: obtaining a sample labeled image data set, the sample labeled image data set including a plurality of sample images, an image label of each sample image, a privacy level of each sample image, and a response time tolerance corresponding to each sample image; for each sample image, inputting the sample image, the privacy level of the sample image, and the response time tolerance corresponding to the sample image into the generative adversarial network, the generator of the generative adversarial network generating an encryption region corresponding to the sample image based on the sample image, the privacy level, and the response time tolerance; performing encryption processing on each sample image based on the encryption region of each sample image to obtain an encrypted image corresponding to each sample image; inputting each sample image and each encrypted image into the generative adversarial network, the discriminator of the generative adversarial network generating a predicted label corresponding to each sample image and each encrypted image, respectively; based on the predicted label corresponding to each sample image and the image label, and the predicted label corresponding to each encrypted image, determining a first loss function value of the discriminator; determine a second loss function value of the generator based on the image label and the predicted label corresponding to each of the encrypted images; if the first loss function value and the second loss function value are both less than a preset loss threshold or a current iteration number reaches a second preset iteration number, it is determined that the generative adversarial network training is completed, and the generator is determined as an encryption region selection model; otherwise, the network parameters of the generator and the network parameters of the discriminator are adjusted, and the execution of inputting, for each sample image, the sample image, the privacy level of the sample image and the response time tolerance corresponding to the sample image into the generative adversarial network is returned.

8. The method of claim 1, wherein, The link information includes a link bandwidth, a link channel gain and a noise power of the target transmission link. The first predicted computing overhead corresponding to execution of each of the to-be-processed tasks by the first device is determined according to the first computing performance information, including: The first predicted time delay and the first predicted energy consumption corresponding to execution of each of the to-be-processed tasks by the first device are determined according to the first computing performance information; The first predicted time delay and the first predicted energy consumption are weighted and summed based on first preset weight information, to obtain the first predicted computing overhead. The second predicted computing overhead corresponding to execution of each of the to-be-processed tasks by the second device is determined according to the second computing performance information and the link information, including: The second predicted time delay and the second predicted energy consumption corresponding to execution of each of the to-be-processed tasks by the second device are determined according to the second computing performance information, the link bandwidth, the link channel gain and the noise power; The second predicted time delay and the second predicted energy consumption are weighted and summed based on second preset weight information, to obtain the second predicted computing overhead.

9. The method of claim 1, wherein, In a case where the target allocation decision is that each of the to-be-processed tasks is executed by the second device, the method further includes: generating an encryption key based on a device identifier of the first device and a set of random numbers maintained by the first device; generating a first encryption sequence and a second encryption sequence according to the encryption key and a chaotic encryption algorithm, the first encryption sequence and the second encryption sequence including the same number of parameters; performing permutation processing on the plurality of to-be-encrypted pixel points based on the first encryption sequence, to obtain an intermediate ciphertext sequence; performing expansion processing on the plurality of to-be-encrypted pixel points based on the second encryption sequence and the intermediate ciphertext sequence, to obtain an encrypted image of each of the to-be-processed tasks.

10. The method of claim 1, wherein, The target allocation decision includes a decision that each of the to-be-processed tasks is executed by the first device, and a decision that each of the to-be-processed tasks is offloaded to the second device and executed by the second device.

11. A computing task allocation apparatus characterized by comprising: The apparatus includes: a first information acquisition module configured to acquire at least one to-be-processed task of a first device, current device state information of the first device and first computing performance information of the first device; The second information acquisition module is configured to acquire second computing performance information of the second device and link information of a target transmission link, the target transmission link being a transmission link between the first device and the second device; The overhead determination module is configured to determine a predicted encryption overhead corresponding to each of the to-be-processed tasks, determine a first predicted computing overhead corresponding to execution of each of the to-be-processed tasks by the first device according to the first computing performance information, and determine a second predicted computing overhead corresponding to execution of each of the to-be-processed tasks by the second device according to the second computing performance information and the link information; The decision module is configured to determine a target allocation decision for each of the to-be-processed tasks based on the predicted encryption overhead, the current device state information, the first predicted computing overhead, the second predicted computing overhead, and a pre-trained decision model, the decision model being a double-network reinforcement learning model. The overhead determination module is specifically configured to acquire at least one task image of each of the to-be-processed tasks, an image label of each of the task images, a privacy level of each of the task images, and a response time tolerance of each of the task images, input each of the task images, the image label of each of the task images, the privacy level of each of the task images, and the response time tolerance of each of the task images into an encryption region selection model to obtain an encryption region in each of the task images, the encryption region selection model being obtained by the second device based on a generative adversarial network, and determine a plurality of to-be-encrypted pixel points according to pixel values of each of the pixel points in the encryption region, and determine a predicted encryption time delay and a predicted encryption energy consumption corresponding to the plurality of to-be-encrypted pixel points as the predicted encryption overhead corresponding to each of the to-be-processed tasks.

12. The apparatus of claim 11, wherein, The device further includes a model training module configured to synchronize network parameters of an online network included in a to-be-trained double-network reinforcement learning model to a target network, the online network and the target network having the same network structure, acquire sample information of a preset number of interaction samples based on an experience replay pool, the sample information of the interaction samples including device state information of a sample device, an allocation decision, a standard reward, and device state information after the allocation decision is executed, the experience replay pool being configured to store sample information of a plurality of interaction samples, input the sample information of the preset number of interaction samples into the target network to obtain a first predicted reward of the target network after the target network executes the allocation decision for each of the interaction samples, input the sample information of the preset number of interaction samples into the online network to obtain a second predicted reward of the online network after the online network executes the allocation decision for each of the interaction samples, determine a loss function of the to-be-trained double-network reinforcement learning model based on the first predicted reward and the second predicted reward, determine a target evaluation index based on the loss function, adjust the network parameters of the online network if the target evaluation index does not meet a training completion condition, and determine that the training is completed if the target evaluation index meets the training completion condition. In a case where the current iteration number reaches a first preset iteration number or training is completed, the network parameters of the online network are synchronized to the target network, and the trained double-network reinforcement learning model after the update parameters is determined as a decision model.

13. The apparatus of claim 12, wherein, The target evaluation index includes a value of the loss function or a gradient value of the loss function; and the training completion condition includes that the value of the loss function is less than a preset loss function threshold or the gradient value of the loss function is less than a preset gradient threshold.

14. The apparatus of claim 12, wherein, The model training module is specifically configured to input sample information of a preset number of interaction samples into the online network, and the online network performs: determining an allocation decision for each interaction sample based on a preset training hyperparameter; and performing the allocation decision for each interaction sample to obtain a second predicted reward corresponding to each interaction sample.

15. The apparatus of claim 14, wherein, The model training module is specifically configured to determine the second predicted reward corresponding to each interaction sample by using the following formula: wherein, is the device state information of the sample device, , n t2 represents the network condition of the sample device at time t, t2 represents the percentage of the remaining power of the sample device at time t, represents the current CPU load occupancy rate of the sample device at time t, and represent the time delay and energy consumption of executing the to-be-processed task locally on the sample device, and represent the total time delay and total energy consumption consumed by offloading the to-be-processed task to the second device, and MASK represents the encryption region selection matrix corresponding to the to-be-processed task, is the encryption time delay corresponding to the encryption region of the to-be-processed task, represents the period of one XOR processing, is the allocation decision, is the second predicted reward after executing the allocation decision , is the calculation overhead of executing the to-be-processed task locally on the sample device, is the calculation overhead corresponding to offloading the to-be-processed task to the second device and executing the to-be-processed task by the second device, and P2(MASK) represents the proportion of the encryption region in the to-be-processed task, , and are corresponding weights, , and are corresponding weights, and are weights corresponding to and P2(MASK).

16. The apparatus of claim 14, wherein, The preset training hyperparameter includes a preset exploration rate; The model training module is specifically configured to, for each interaction sample, generate a random number based on device state information of a sample device corresponding to the interaction sample; In response to the random number being not less than the preset exploration rate, determining the allocation decision for the interaction sample based on a target probability and a greedy model.

17. An electronic device, comprising: comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the method of any one of claims 1-10 when executing the program.

18. A storage medium containing computer-executable instructions, wherein: The computer executable instructions, when executed by the computer processor, are used to perform the method of any one of claims 1-10.

Images