Perception task execution method, device and equipment

Abstract

The invention provides a perception task execution method, device and equipment, and the method is applied to an edge cloud cooperation system, and the method comprises the steps: a cloud end receives an initial perception task model and a plurality of perception data samples uploaded by a user, carries out the training of the initial perception task model based on the perception data samples, and carries out the execution of a perception task; the multi-task model comprises K parameter separation model branches, the K parameter separation model branches are respectively used for reasoning sensing data under K types of task distribution, and the cloud side sends the multi-task model and the L average task distribution representations to the side end. And the side end selects a to-be-used parameter separation model branch based on the task distribution representation of the to-be-reasoned sensing data and the L average task distribution representations, and reasones the to-be-reasoned sensing data based on the to-be-used parameter separation model branch. According to the method and the device, the perception data distributed by various different tasks can be effectively reasoned.

Description

[0001] This disclosure claims priority to Chinese patent application No. 202310847808.2 filed on July 11, 2023, with invention name “Method, apparatus, computing device cluster and storage medium for task execution”, the entire contents of which are incorporated by reference in this disclosure. Technical Field

[0002] The present disclosure relates to the field of cloud technology, and in particular to a method, device and equipment for executing a perception task. Background Art

[0003] With the development of IoT technology, more and more edge devices are beginning to perform various perception tasks. Edge devices include smartphones, drones, robots, etc., and perception tasks include visual semantic recognition, speech recognition, natural language processing, etc. The edge performs perception tasks through the perception task model. Since the edge is closer to the user, in order to improve the efficiency of perception task execution, the perception task model is generally deployed on the edge. However, the computing power of the edge is limited and the training of the perception task model cannot be achieved. Compared with the edge, the cloud has a larger amount of computing power, so the training of the perception task model is usually implemented by the cloud.

[0004] In the related art, the perception task model can usually effectively reason about the perception data under the specified task distribution, but the reasoning effect is poor for the perception data under other task distributions. Task distribution refers to the environmental information and device information when collecting perception data. Environmental information includes weather, location, etc., and device information includes the device mode, image resolution, etc. For example, when the perception task is visual perception semantic recognition, the perception task model in the related art can only effectively recognize images taken at the front door of the park on a sunny day, and the recognition effect is poor for images taken in the park garden on a cloudy day. Summary of the invention

[0005] The present disclosure provides a method, device and apparatus for executing a perception task, which can execute a variety of perception tasks under different task distributions. The corresponding technical solutions are as follows:

[0006] In a first aspect, a method for executing a perception task is provided, and the method is applied to an edge-cloud collaborative system, and the edge-cloud collaborative system includes an edge and a cloud. The method includes: the cloud receives an initial perception task model and multiple perception data samples uploaded by a user, wherein the multiple perception data samples belong to K-type task distributions, and K is an integer greater than 1. The cloud trains the initial perception task model based on the multiple perception data samples to obtain a multi-task model and L average task distribution representations, wherein the multi-task model includes K parameter separation model branches, and the K parameter separation model branches are respectively used to infer the perception data under the K-type task distribution, and the L average task distribution representations and the K parameter separation model branches are in a many-to-one or one-to-one relationship. The cloud sends the multi-task model and the K average task distribution representations to the edge. The edge collects the perception data to be inferred, obtains the task distribution representation of the perception data to be inferred, and selects a parameter separation model branch to be used from the K parameter separation model branches based on the task distribution representation of the perception data to be inferred and the K average task distribution representations. The edge infers the perception data to be inferred based on the parameter separation model branch to be used.

[0007] In the technical solution provided by the present disclosure, the edge and cloud collaborate to perform perception tasks, and the user uploads the initial perception task model and multiple perception data samples to the cloud. The cloud trains the initial perception task model based on the multiple perception data samples uploaded by the user to obtain a multi-task model and L average task distribution representations, wherein the multi-task model includes K parameter separation model branches, and the K parameter separation model branches are respectively used to infer the perception data under K types of task distribution. Then, the cloud sends the multi-task model and the K average task distribution representations to the edge, which is deployed and infers the perception data to be inferred. During inference, the edge adaptively selects the parameter separation model branch suitable for executing the current perception task from the K parameter separation model branches based on the task distribution representation of the perception data to be inferred and the L average task distribution representations to infer the perception data to be inferred. It can be seen that through the above scheme, the task distribution representation of the perception task to be inferred can be automatically identified, and the parameter separation model branch suitable for executing the current perception task can be adaptively selected. In this way, perception tasks under a variety of different task distributions can be executed.

[0008] In a possible implementation, in order to enable the user to intuitively know the task distribution to which the currently executed perception task belongs, the cloud can also determine the interpretable task distribution label group corresponding to each parameter separation model branch. Accordingly, the processing can be as follows:

[0009] Based on multiple perception data samples, the cloud side trains the initial perception task model to obtain a multi-task model, L average task distribution representations, and K interpretable task distribution label groups, where the K interpretable task distribution label groups correspond one to one to the K parameter separation model branches.

[0010] In the technical solution provided by the present disclosure, the task distribution interpretable label includes at least one of an environment label and a collection device label. The environment label can indicate the environment when the perception data is collected, such as weather, location, light intensity, etc., and the collection device label can indicate the collection device information for collecting the perception data, such as the mode of the collection device, the resolution of the image taken by the collection device, etc.

[0011] In a possible implementation, in order to obtain a parameter separation model branch that can perform perception tasks under different task distributions, the cloud can first cluster the perception data samples uploaded by the user, and then train a parameter separation model branch based on each sample set obtained by clustering. Correspondingly, the processing can be as follows:

[0012] The cloud obtains the task distribution representation of each of the multiple perception data samples, and obtains the interpretable task distribution label of the perception data sample. Based on the task distribution representation of the multiple perception data samples, the multiple perception data samples are clustered into K sample sets, wherein the K sample sets correspond to K types of task distributions, and each sample set includes at least one perception data sample. For each of the K sample sets, the cloud trains the last M layers in the initial perception task model based on the sample set to obtain a parameter separation model branch, and uses the mean of the task distribution representation of each perception data sample in the sample set as the average task distribution representation corresponding to the trained parameter separation model branch, and uses the set of interpretable task distribution labels of each perception data sample in the sample set as the interpretable task distribution label corresponding to the trained parameter separation model branch. The cloud combines the first N layers of the initial perception task model and the K parameter separation model branches obtained by training to form a multi-task model, wherein the initial perception task model consists of the first N layers and the last M layers, and in the initial perception task model, the first N layers are located before the last M layers.

[0013] In one possible implementation, in order to automatically extract task distribution representation of perception data, the process may be as follows:

[0014] The perception data samples are input into the first N layers of the initial perception task model to obtain the intermediate feature data output by each of the first N layers. Based on the intermediate feature data output by each of the first N layers, the task distribution representation of the perception data samples is calculated.

[0015] In one possible implementation, the process of obtaining interpretable task distribution labels for perceptual data samples may be as follows:

[0016] The perception data is obtained from the correspondence between the stored perception data and the task distribution interpretable label, and for each perception data obtained, the task distribution representation of the perception data is obtained. The similarity between the task distribution representation of the perception data sample and the task distribution representation of each perception data in the correspondence is calculated, and the task distribution interpretable label corresponding to the perception data corresponding to the maximum similarity is used as the task distribution interpretable label corresponding to the perception data sample.

[0017] In a possible implementation, when performing a perception task, the process of edge-adaptive selection of parameter separation model branches may be as follows:

[0018] The edge calculates the similarity between the task distribution representation of the perception data to be inferred and the L average task distribution representations, and selects the parameter separation model branch corresponding to the average task distribution representation corresponding to the maximum similarity as the parameter separation model branch to be used.

[0019] In a possible implementation, the method further includes:

[0020] The cloud receives a model update request sent by a user, wherein the model update request carries a target average task distribution representation and a target perception data sample, wherein the target perception data sample is perception data under a target task distribution, and the target task distribution is a task distribution indicated by an interpretable label of a task distribution corresponding to the target average task distribution representation;

[0021] The cloud end trains a parameter separation model branch corresponding to the target average task distribution representation in the multi-task model based on the target perception data sample.

[0022] In a second aspect, a method for executing a perception task is provided, the method being applied to a cloud end in an edge-cloud collaborative system, the edge-cloud collaborative system also including an edge end, the method comprising:

[0023] Receiving an initial perception task model and a plurality of perception data samples uploaded by a user, wherein the plurality of perception data samples belong to K-type task distributions, where K is an integer greater than 1;

[0024] Based on the multiple perception data samples, the initial perception task model is trained to obtain a multi-task model and K average task distribution representations, wherein the multi-task model includes K parameter separation model branches, the K parameter separation model branches are respectively used to infer the perception data under the K types of task distributions, and the K parameter separation model branches correspond to the K average task distribution representations one by one;

[0025] The multi-task model and the K average task distribution representations are sent to the edge.

[0026] In a third aspect, a method for executing a perception task is provided, the method being applied to an edge in an edge-cloud collaborative system, the edge-cloud collaborative system also including a cloud, the method comprising:

[0027] Receiving the multi-task model and the K average task distribution representations sent by the cloud, wherein K is an integer greater than 1, the multi-task model includes K parameter separation model branches, the K parameter separation model branches are respectively used to infer the perception data under the K types of task distributions, and the K parameter separation model branches correspond to the K average task distribution representations one by one;

[0028] Collect perception data to be inferred;

[0029] Obtaining a task distribution representation of the perception data to be inferred;

[0030] Based on the task distribution representation of the perception data to be inferred and the K average task distribution representations, selecting a parameter separation model branch to be used from the K parameter separation model branches;

[0031] The to-be-inferred perception data is inferred based on the to-be-used parameter separation model branch.

[0032] In a fourth aspect, a device for executing a perception task is provided, the device being applied to a cloud end in an edge-cloud collaborative system, the edge-cloud collaborative system also including an edge end, the device comprising:

[0033] A task mining training module is used to receive an initial perception task model and multiple perception data samples uploaded by a user, wherein the multiple perception data samples belong to K-type task distributions, and K is an integer greater than 1; based on the multiple perception data samples, the initial perception task model is trained to obtain a multi-task model and L average task distribution representations, wherein the multi-task model includes K parameter separation model branches, and the K parameter separation model branches are respectively used to infer the perception data under the K-type task distribution;

[0034] A shared storage module is used to send the multi-task model and the L average task distribution representations to the edge.

[0035] In a fifth aspect, a device for executing a perception task is provided, the device being applied to an edge in an edge-cloud collaborative system, the edge-cloud collaborative system also including a cloud, and the device comprising:

[0036] A task distribution representation calculation module is used to receive the multi-task model and the L average task distribution representations sent by the cloud, wherein L and K are integers greater than 1, the multi-task model includes K parameter separation model branches, the K parameter separation model branches are respectively used to infer the perception data under the K types of task distribution, and the K parameter separation model branches correspond to the L average task distribution representations one by one; collect perception data to be inferred; obtain the task distribution representation of the perception data to be inferred;

[0037] An adaptive model branch selection reasoning module is used to select a parameter separation model branch to be used from the K parameter separation model branches based on the task distribution representation of the perception data to be inferred and the L average task distribution representations; and to reason about the perception data to be inferred based on the parameter separation model branch to be used.

[0038] In a sixth aspect, an edge terminal is provided, wherein the edge terminal includes a processor and a memory;

[0039] The processor is used to execute instructions stored in the memory of the edge so that the edge executes the method for executing the perception task as described in the third aspect above.

[0040] In a seventh aspect, a cloud is provided, wherein the cloud is a computer device cluster, the computer device cluster includes at least one computing device, and each computing device includes a processor and a memory;

[0041] The processor of the at least one computing device is used to execute instructions stored in the memory of the device, so that the computing device cluster executes the method for executing the perception task as described in the second aspect above.

[0042] In an eighth aspect, a computer program product comprising instructions is provided. When the instructions are executed by an edge, the edge executes the method for executing the perception task as described in the third aspect above.

[0043] In a ninth aspect, a computer program product comprising instructions is provided. When the instructions are executed by a cloud, the cloud executes the method for executing the perception task as described in the second aspect above.

[0044] In the tenth aspect, a computer-readable storage medium is provided, comprising computer program instructions. When the computer program instructions are executed by an edge, the edge executes the method for executing the perception task as described in the third aspect above.

[0045] In the eleventh aspect, a computer-readable storage medium is provided, comprising computer program instructions. When the computer program instructions are executed by an edge, the edge executes the method for executing the perception task as described in the second aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is a structural diagram of a multi-task model provided by an embodiment of the present disclosure;

[0047] Figure 2 It is a flowchart of a method for executing a perception task provided by an embodiment of the present disclosure;

[0048] Figure 3 It is a flowchart of a method for executing a perception task provided by an embodiment of the present disclosure;

[0049] Figure 4 is a structural diagram of a multi-task model provided by an embodiment of the present disclosure;

[0050] Figure 5 is a structural diagram of a device for executing a perception task provided by an embodiment of the present disclosure;

[0051] Figure 6 It is a schematic diagram of the architecture of a lifelong learning framework provided by an embodiment of the present disclosure;

[0052] Figure 7 is a schematic diagram of the structure of a computing device provided by an embodiment of the present disclosure;

[0053] Figure 8 is a schematic diagram of the structure of a computing device provided by an embodiment of the present disclosure;

[0054] Fig. 9 is a schematic diagram of a computer device cluster structure provided by an embodiment of the present disclosure;

[0055] Fig.10 It is a schematic diagram of a computer device cluster structure provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0056] In order to make the objectives, technical solutions and advantages of the present disclosure more clear, the embodiments of the present disclosure will be further described in detail below with reference to the accompanying drawings.

[0057] Some terms involved in the embodiments of the present disclosure are explained below.

[0058] Edge-Cloud Collaboration : A system computing mode based on the collaboration between the edge and the cloud to complete the algorithm process, such as the collaboration between the edge and the cloud to complete the perception task. In the present disclosure, the multi-task model is trained by the cloud, the trained multi-task model is deployed by the edge, and the edge uses the multi-task model to perform the perception task.

[0059] Cloud: A cloud service cluster used to provide remote cloud services such as computing power and storage. In this disclosure, the cloud is used to train multi-task models.

[0060] Edge : Devices close to the user side, such as computing devices such as IoT terminals and edge servers. IoT terminals can be smartphones, drones, quadruped robots, etc. In this disclosure, the edge is used to deploy a multi-task model and use the multi-task model to perform perception tasks.

[0061] Perception Task :Intelligent computing tasks based on sensor input. Sensors can be visual, speech, or communication sensors. Correspondingly, perception tasks can be visual perception tasks, speech perception tasks, or wireless perception tasks. Visual perception tasks, such as road semantic recognition. Speech perception tasks, such as timbre recognition. Wireless perception tasks, such as human behavior detection based on wireless signals.

[0062] Perception Data :Data collected by sensors as input for perception tasks. For visual perception tasks, the perception data can be RGB (red green blue) images or depth images. For speech perception tasks, the perception data can be audio data. For wireless perception tasks, the perception data can be wireless communication signals.

[0063] Task distribution : refers to the environment and equipment for collecting sensory data. Environmental information may include location, weather, lighting, etc. Equipment information may include the modality of the device for collecting sensory data, the resolution of the image taken by the device, etc. The modality may include the modality for collecting RGB images and the modality for collecting depth images.

[0064] Task distribution representation : A vector used to characterize the distribution of tasks.

[0065] Task distribution interpretable labels : A label used to describe task distribution, which is understandable to humans. The task distribution includes environmental information and device information for collecting perception data. The interpretable label of the task distribution here can refer to the environmental label used to describe the environmental information and the device label used to describe the device information.

[0066] The embodiment of the present disclosure provides a method for executing a perception task, which can be applied to an edge-cloud collaborative system, which includes an edge and a cloud, wherein the cloud is used to train a multi-task model, the edge downloads and deploys the trained multi-task model, and uses the trained multi-task model to execute the perception task. Figure 1As shown, the multi-task model proposed in the embodiment of the present disclosure includes a weight sharing layer and multiple parameter separation model branches. In the multi-task model, the weight sharing layer is located before the parameter separation model branch, and different parameter separation model branches are used to infer the perception data of different task distributions. After the perception data is collected, it is first input into the weight sharing layer, and the task distribution representation of the perception data is calculated based on the intermediate feature data extracted by the weight sharing layer. Then, among the multiple parameter separation model branches, the parameter separation model branch that matches the task distribution representation is selected, and the intermediate feature data is input into the selected parameter separation model to continue reasoning, and the reasoning result is output. It can be seen that through the multi-task model provided by the embodiment of the present disclosure, the reasoning of perception data under different task distributions can be achieved.

[0067] The method for executing the perception task provided in the embodiment of the present disclosure can be divided into two parts: cloud training and edge reasoning. The cloud training and edge reasoning are described below. Figure 2 ,The processing process of cloud training can include the following steps:

[0068] Step 210: The cloud obtains an initial perception task model and multiple perception data samples.

[0069] In implementation, there are many ways for the cloud to obtain the initial perception task model, and several of them are described below.

[0070] Method 1: Receive the initial perception task model uploaded by the user.

[0071] In this way, users can customize the uploaded initial perception task model according to their needs. The initial perception task model is a neural network model. For example, if the user is performing the same visual perception task, the initial perception task model can be ResNet18.

[0072] Method 2: Obtain an initial perception task model from the perception task models stored in the cloud and the perception task models publicly available on the Internet.

[0073] In this second method, the user can send a perception task model acquisition request to the cloud, and the perception task model acquisition request can carry the indication information of the perception task, for example, the perception task can be a visual perception task, a speech perception task, a wireless perception task, etc. Correspondingly, after receiving the perception task acquisition request, the cloud can obtain the perception task model that can be used to perform the perception task indicated by the above indication information from the stored perception task models and the perception task models publicly available on the Internet as the initial perception task model. For example, if the perception task indicated by the indication information is a visual perception task, the cloud can obtain ResNet18 as the initial perception task model.

[0074] There are also multiple ways to obtain perception data samples in the cloud, some of which are described below.

[0075] Method 1: Receive the perception data samples uploaded by the user side.

[0076] In this way, users can collect perception data under different task distributions according to their own needs as perception data samples, and upload the perception data samples to the cloud. For example, if the user wants to perform road semantic recognition in the visual perception task, the user can collect RGB images of the road at the front door of the park on a rainy day, RGB images of the road at the front door of the park on a sunny day, depth images of the road at the front door of the park on a rainy day, depth images of the road at the front door of the park on a sunny day, RGB images of the road at the garden on a rainy day, RGB images of the road at the garden on a sunny day, depth images of the road at the garden on a rainy day, depth images of the road at the garden on a sunny day, and other road images under various task distributions as perception data samples.

[0077] Method 2: Obtain data samples from the perception data stored in the cloud and the perception data publicly available on the Internet.

[0078] In this second method, the user can send a perception data sample acquisition request to the cloud, and the perception data sample acquisition request can carry the indication information of the perception task, for example, the perception task is the road semantic recognition in the visual perception task. Accordingly, after receiving the perception data sample acquisition request, the cloud can obtain the perception data of the perception task indicated by the above indication information under various task distributions from the stored perception data and the perception data publicly available on the Internet as the perception data sample.

[0079] After acquiring the perception data samples, the perception data samples can be annotated by an annotator, or the cloud can automatically annotate the perception data samples. For example, if the perception task is road semantic recognition and the perception data samples are images, the annotation of the perception data samples can be semantic segmentation of the images and the corresponding semantic segmentation labels. The semantic segmentation labels can be used to indicate the types of objects obtained by semantic segmentation, such as vehicles, pedestrians, obstacles, etc.

[0080] In a possible implementation, after obtaining the initial perception task model and the perception data samples in the cloud, the initial perception task model can be initially trained using the perception data samples as input samples and the annotation results of the perception data samples as output samples. The initial training can use the gradient descent method in supervised learning. Taking the perception task of road semantic recognition as an example, the annotation results can be semantic segmentation labels.

[0081] In the case where the initial perception task model has been initially trained, in subsequent steps, the initial perception task model used may refer to the perception task model that has undergone initial training.

[0082] Step 220: The cloud uses the N layers in the initial perception task model as weight sharing layers.

[0083] In implementation, the intermediate feature data output by the N layers in the initial perception task model can reflect the task distribution of the perception data to a certain extent. Therefore, the N layers in the initial perception task model can be used as weight sharing layers without parameter adjustment in the subsequent training process.

[0084] The N layers of the initial perception task model can be the first N layers of the initial perception task model, or the second to N+1 layers of the initial perception task model. Of course, there are other possibilities, which are not listed here. The specific selection of the N layers of the initial perception task can be configured by the technicians according to the characteristics of the initial perception task model. For example, if the initial perception task model is ResNet18, the value of N can be 2, and the N layers of the initial perception task model can be the first two layers of the initial perception task model, which are convolutional layers.

[0085] Step 230: For each perception data sample, the cloud inputs the perception data sample into the weight sharing layer to obtain the intermediate feature data output by each of the N layers included in the weight sharing layer, and calculates the task distribution representation of the perception data sample based on the intermediate feature data output by each of the N layers.

[0086] In implementation, for each perception data sample obtained, the cloud inputs the perception data sample into the initial perception task model, and then obtains the intermediate feature data output by each of the N layers included in the weight sharing layer. For the intermediate feature data output by each of the N layers, the statistical results of each element in the intermediate feature data are calculated, and the statistical results corresponding to the N intermediate feature data are respectively composed into a vector as a task distribution representation of the perception data sample. Among them, the statistical results may include at least one of the mean, variance, and standard deviation. The following takes the example of the statistical results including the mean and variance, combined with the following formula, to illustrate the task distribution representation of the calculated perception data sample.

[0087]

[0088]

[0089] f(r)=[,01,…μN,0v]

[0090] Among them, μ i Represents the mean value of each element in the intermediate feature data output by the i-th layer in the N-layer, mi Indicates the number of elements in the intermediate feature data output by the i-th layer in the N-layer, p j represents the value of the jth element in the intermediate feature data output by the i-th layer in the N-th layer, σ i It represents the variance of each element in the intermediate feature data output by the Nth layer, and f(x) represents the task distribution representation of the perception data sample.

[0091] Step 240: The cloud clusters the multiple perception data samples into K sample sets based on the task distribution representation of the multiple perception data samples.

[0092] Wherein, K is a positive integer, each sample set includes at least one perception data sample, and the perception data samples included in different sample sets have different task distributions.

[0093] In implementation, after obtaining the task representation of each perception data sample, the cloud can cluster multiple perception data samples into K sample sets based on the task representation of each perception data sample. The value of K can be configured according to actual needs. For example, the value of K is 5. Clustering can be implemented using clustering algorithms such as K-Means++ unsupervised clustering algorithm and K-Means clustering algorithm. The following is an exemplary clustering method for illustration. The clustering method may include the following steps:

[0094] S1. Get the initialization constant K, randomly initialize and generate K cluster centers, where the cluster center is a vector with the same dimension as the task distribution representation.

[0095] S2. For each sensed data sample, calculate the distance between the sensed data sample and each cluster center, determine the target cluster center corresponding to the minimum distance, and divide the sensed data sample into the sample set corresponding to the target cluster center. The sample set can also be called a cluster.

[0096] S3. After all the perception data are divided into sample sets, for each sample set, the task distribution representation of each perception data sample in the sample set is averaged to obtain the average task distribution representation of the sample set, and the cluster center of the sample set is updated to the average task distribution representation.

[0097] S4. Repeat steps S2 and S3 until the cluster centers in all sample sets no longer change, and clustering is completed.

[0098] After clustering, K sample sets can be obtained, and each sample set corresponds to an average task distribution representation. In a possible implementation, for each sample set, at least one perception data sample can be randomly selected from the perception data samples included in the sample set, and the task distribution representation of the selected perception data sample can also be used as the average task distribution representation of the sample set. In this way, one sample set can correspond to multiple average task distribution representations.

[0099] Step 250: The cloud expands the last M layers of the initial perception task model to obtain K parameter separation model branches to be trained.

[0100] Among them, the last M layers refer to the first layer of the weight sharing layer to the last layer of the initial perception task model.

[0101] In implementation, the cloud copies the last M layers of the initial perception task model to obtain K parameter separation model branches to be trained, and each parameter separation model branch to be trained is the last M layers of the initial perception task model. This is explained below with an example.

[0102] like Figure 3 As shown in the figure, the initial perception task module consists of N+M layers. The first N layers are used as weight sharing layers, and the last M layers are copied and expanded to obtain K parameter separation model branches to be trained.

[0103] Step 260: The cloud trains K parameter separation model branches to be trained based on the perception data samples included in the K sample sets.

[0104] In implementation, the K sample sets are respectively recorded as sample set 1, sample set 2, ... sample set K, and the K parameter separation model branches to be trained are respectively recorded as parameter separation model branch 1, parameter separation model branch 2, ... parameter separation model branch K. The parameter separation model branch 1 is trained using the perceptual data samples included in the sample set 1, the parameter separation model branch 2 is trained using the perceptual data samples included in the sample set 2, and so on.

[0105] Take the training of parameter separation model branch 1 using the perception data samples included in sample set 1 as an example. For each perception data sample included in sample set 1, the perception data sample is used as an input sample and input into the weight sharing layer. The intermediate feature data output by the last layer of the weight sharing layer is input into the parameter separation model branch 1 to obtain the inference result output by the parameter separation model branch 1. The calibration result corresponding to the perception data sample is used as the output sample. According to the inference result output by the parameter separation model branch 1 and the calibration result corresponding to the perception data sample, the loss value is calculated, and the parameters in the parameter separation model 1 are adjusted according to the loss value. In addition, only an exemplary training method is given here, and other supervised learning training methods can also be used for training. The embodiments of the present disclosure do not limit the specific training method.

[0106] After using a sample set to train a parameter separation model branch, the average task distribution representation of the sample set can be used as the average task distribution representation corresponding to the parameter separation model branch, which is used as the basis for selecting the parameter separation model branch in the edge inference stage.

[0107] Step 270: The cloud separates the weight sharing layer and the K trained parameter model branches to form a multi-task model.

[0108] In implementation, after the training of K parameter separation model branches to be trained is completed, the weight sharing layer and the K trained parameter separation model branches are combined into a multi-task model, and the weight sharing layer is located before the K trained parameter separation model branches.

[0109] In a possible implementation, the interpretable task distribution label corresponding to each perception data sample can also be determined. Specifically, the process can be as follows:

[0110] The correspondence between the perception data and the interpretable labels of the task distribution can be stored in the explainable knowledge database of the cloud. For each perception data in the correspondence, the cloud can input the perception data into the above-mentioned weight sharing layer to obtain the intermediate feature data, and calculate the task distribution representation of the perception data based on the intermediate feature data. The specific calculation method is the same as the calculation method in the above-mentioned step 230, which will not be repeated here. For each perception data sample, the similarity between the task distribution representation of the perception data sample and the task distribution representation of each perception data in the above-mentioned correspondence is calculated, and the task distribution interpretable label corresponding to the perception data corresponding to the maximum similarity is used as the task distribution interpretable label corresponding to the perception data sample. The similarity here can be expressed by the wasserstein distance.

[0111] On this basis, for each of the above sample sets, the task distribution interpretable labels corresponding to each perception data sample in the sample set can be deduplicated to obtain the task distribution interpretable labels corresponding to the sample set, that is, the task distribution interpretable labels corresponding to the parameter separation model branch trained using the sample set, that is, the task distribution interpretable labels corresponding to the average task distribution representation corresponding to the parameter separation model branch trained with the sample set.

[0112] Step 280: The cloud sends the multi-task model to the edge.

[0113] In implementation, after the cloud completes training of the multi-task model, it can be downloaded and deployed by the edge. When the edge downloads the multi-task model, it can also download the average task distribution representation corresponding to each parameter separation model branch in the multi-task model and the task distribution interpretable label corresponding to each average task distribution representation. After deployment, the edge can use the task model to perform perception tasks. The following is an explanation of edge reasoning, see Figure 4 , edge reasoning can include the following processing steps:

[0114] Step 301: The edge collects the perception data to be inferred.

[0115] In practice, for different perception tasks, the perception data to be inferred collected by the edge is also different. For example, if the perception task is road semantic recognition, the edge can be a quadruped robot, and the quadruped robot can collect at least one of the RGB image and depth image of the road through a visual sensor as the perception data to be inferred.

[0116] Step 302: The edge inputs the perception data into the weight sharing layer of the multi-task model to obtain the intermediate feature data output by the weight sharing layer, and calculates the task distribution representation of the perception data based on the intermediate feature data output by the weight sharing layer.

[0117] In implementation, the specific processing of calculating the task distribution representation of the perception data in step 302 is similar to the specific processing of calculating the task representation of the perception data samples in the above-mentioned step 230. The only difference is that the processing of step 302 is performed by the edge, and the processing of step 230 is performed by the cloud. The processing will not be repeated here.

[0118] Step 303: The edge determines, from the average task distribution representations corresponding to the K parameter separation model branches, a first average task distribution representation having the highest similarity to the task distribution representation of the perception data to be inferred.

[0119] In the implementation, the edge calculates the similarity between the task distribution representation of the perception data to be inferred and the average task distribution representation corresponding to each parameter separation model branch, and determines the first average task distribution representation corresponding to the maximum similarity. The similarity here can be expressed by the wasserstein distance.

[0120] Step 304: The edge inputs the intermediate feature data output by the last layer in the weight sharing layer into the parameter separation model branch corresponding to the first average task distribution representation to obtain the inference result of the perception data to be inferred.

[0121] During implementation, the edge determines the parameter separation model branch corresponding to the first average task distribution representation, and inputs the intermediate feature data output by the last layer in the weight sharing layer into the parameter separation model branch corresponding to the first average task distribution representation to obtain the inference result of the perception data to be inferred.

[0122] In a possible implementation, after determining the first average task distribution representation, the first task distribution interpretable label corresponding to the first average task distribution representation can also be determined. In this case, if the user finds that the inference result of the multi-task model for the perceived data is incorrect, the perceived data sample under the first task distribution indicated by the first task distribution interpretable label can be collected. And send a model update request to the cloud, carrying the perceived data sample under the first task distribution and the first average task distribution representation in the model update request. Accordingly, after receiving the model update request, the cloud uses the perceived data sample under the first task distribution to train the parameter separation model corresponding to the first average task distribution representation. In this way, the parameter separation model branch that needs to be updated can be quickly located, thereby guiding the user to collect the perceived data samples under a specific task distribution, achieving rapid model adaptation and updating, and reducing labor and time costs.

[0123] The present disclosure provides a device for executing a perception task, such as Figure 5 As shown, the device 500 can be applied to the cloud in the edge-cloud collaborative system. The device 500 includes a task mining training module 501 and a shared storage module 502, wherein:

[0124] The task mining training module 501 is used to receive an initial perception task model and multiple perception data samples uploaded by a user, wherein the multiple perception data samples belong to K-type task distributions, and K is an integer greater than 1; based on the multiple perception data samples, the initial perception task model is trained to obtain a multi-task model and L average task distribution representations, wherein the multi-task model includes K parameter separation model branches, and the K parameter separation model branches are respectively used to infer the perception data under the K-type task distribution; specifically, the task mining training module 501 can implement the processing of the above steps 210-270 and the processing of optional steps thereof.

[0125] The shared storage module 502 is used to send the multi-task model and the L average task distribution representations to the edge. Specifically, the shared storage module 502 can implement the processing of step 280 above.

[0126] The present disclosure provides a device for executing a perception task, such as Figure 5 As shown, the device 600 can be applied to the edge in the edge-cloud collaborative system. The device 600 includes a task distribution representation calculation module 601 and an adaptive model branch selection reasoning module 602, wherein:

[0127] The task distribution representation calculation module 601 is used to receive the multi-task model sent by the cloud and the L average task distribution representations, wherein L and K are integers greater than 1, and the multi-task model includes K parameter separation model branches, and the K parameter separation model branches are respectively used to infer the perception data under the K types of task distribution, and the K parameter separation model branches correspond to the L average task distribution representations one by one; collect the perception data to be inferred; obtain the task distribution representation of the perception data to be inferred; specifically, the task distribution representation calculation module 601 can implement the processing of the above steps 301 and 302.

[0128] The adaptive model branch selection reasoning module 602 is used to select a parameter separation model branch to be used from the K parameter separation model branches based on the task distribution representation of the perception data to be inferred and the L average task distribution representations; and to reason about the perception data to be inferred based on the parameter separation model branch to be used. Specifically, the adaptive model branch selection reasoning module 602 can implement the processing of steps 303 and 304 above.

[0129] Among them, the task mining training module 501, the shared storage module 502, the task distribution representation calculation module 601 and the adaptive model branch selection reasoning module 602 can all be implemented by software, or can be implemented by hardware. Exemplarily, the implementation of the task mining training module 501 is introduced below, taking the task mining training module 501 as an example. Similarly, the implementation of the shared storage module 502, the task distribution representation calculation module 601 and the adaptive model branch selection reasoning module 602 can refer to the implementation of the A module.

[0130] As an example of a software functional unit, the task mining training module 501 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the above-mentioned computing instance may be one or more. For example, the task mining training module 501 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code can be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code can be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple data centers with close geographical locations. Among them, usually a region can include multiple AZs.

[0131] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0132] As an example of a hardware functional unit, the task mining training module 501 may include at least one computing device, such as a server, etc. Alternatively, the task mining training module 501 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0133] The multiple computing devices included in the task mining training module 501 can be distributed in the same region or in different regions. The multiple computing devices included in the A module can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the task mining training module 501 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0134] It should be noted that, in other embodiments, the task mining training module 501 can be used to execute any step executed in the cloud in the execution method of the perception task, the shared storage module 502 can be used to execute any step executed in the cloud in the execution method of the perception task, the task distribution representation calculation module 601 can be used to execute any step executed on the edge in the execution method of the perception task, and the adaptive model branch selection reasoning module 602 can be used to execute any step executed on the edge in the execution method of the perception task. The steps that the task mining training module 501, the shared storage module 502, the task distribution representation calculation module 601 and the adaptive model branch selection reasoning module 602 are responsible for implementing can be specified as needed, and the task mining training module 501, the shared storage module 502, the task distribution representation calculation module 601 and the adaptive model branch selection reasoning module 602 respectively implement different steps in the execution method of the perception task to realize all the functions of the execution device of the perception task.

[0135] In one possible implementation, Figure 5 The modules in can be integrated into edge cloud collaborative computing frameworks such as lifelong learning frameworks. Figure 6, the lifelong learning framework includes a global management component (Global Manager), a local control component (Local controller) in the cloud, a lifelong training component (Lifelong Training Worker), a lifelong learning knowledge base, a labeling system, a shared storage service, a local controller on the edge, and a lifelong reasoning component (Lifelong Inference Worker). Among them, the Global Manager includes a data set management module and a lifelong learning control module (Lifelong learning controller). The Lifelong Training Worker originally includes a cloud / edge server training module. In the disclosed embodiment, a task mining training module is integrated in the Lifelong Training Worker. The lifelong learning knowledge base includes a knowledge base management module, an unknown task processing module, and an interpretable knowledge database. The interpretable knowledge database is used to store the correspondence between perception data and interpretable labels of task distribution. The Lifelong Inference Worker originally includes an edge reasoning module and an unknown task identification module. In the disclosed embodiment, a task distribution representation calculation module and an adaptive model branch selection reasoning module are integrated in the Lifelong Inference Worker.

[0136] The present disclosure also provides a computing device 200, which may be an edge terminal in an edge-cloud collaborative system. Figure 7 As shown, computing device 200 includes: bus 202, processor 204, memory 206 and communication interface 208. Processor 204, memory 206 and communication interface 208 communicate through bus 202. Computing device 200 can be cloud or edge. It should be understood that the present disclosure does not limit the number of processors and memories in computing device 200.

[0137] The bus 202 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 The bus 202 is represented by only one line, but it does not mean that there is only one bus or one type of bus. The bus 202 may include a path for transmitting information between various components of the computing device 200 (eg, the memory 206, the processor 204, and the communication interface 208).

[0138] The processor 204 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0139] The memory 206 may include a volatile memory, such as a random access memory (RAM). The memory 206 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0140] The memory 206 stores executable codes, and the processor 204 executes the executable codes to respectively implement the functions of the aforementioned task distribution representation calculation module 601 and the adaptive model branch selection reasoning module 602, thereby implementing the edge execution processing in the execution method of the perception task. That is, the memory 206 stores instructions for executing the execution method of the perception task.

[0141] The communication interface 208 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 200 and other devices or a communication network.

[0142] The present disclosure also provides a computing device 100, which may be a cloud in an edge-cloud collaborative system. Figure 8 As shown, the computing device 100 includes: a bus 102, a processor 104, a memory 106, and a communication interface 108. The processor 104, the memory 106, and the communication interface 108 communicate through the bus 102. The computing device 100 can be a cloud or an edge. It should be understood that the present disclosure does not limit the number of processors and memories in the computing device 100.

[0143] The bus 102 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8The bus 102 is represented by only one line, but it does not mean that there is only one bus or one type of bus. The bus 102 may include a path for transmitting information between various components of the computing device 100 (eg, the memory 106, the processor 104, and the communication interface 108).

[0144] The processor 104 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0145] The memory 106 may include a volatile memory, such as a random access memory (RAM). The memory 106 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0146] In the case where the computing device 100 is an edge, the memory 106 stores executable codes, and the processor 104 executes the executable codes to respectively implement the functions of the aforementioned task mining training module 501 and the shared storage module 502, thereby implementing the cloud processing in the method for executing the perception task. That is, the memory 106 stores instructions for executing the method for executing the perception task.

[0147] When the computing device 100 is a cloud computing device, the memory 106 stores executable codes, and the processor 104 executes the executable codes to respectively implement the functions of the aforementioned task mining training module 501 and the shared storage module 502, thereby implementing the training method of the multi-task model. That is, the memory 106 stores instructions for executing the training method of the multi-task model.

[0148] The communication interface 108 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 100 and other devices or a communication network.

[0149] The present disclosure also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device may be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device may also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0150] like Fig. 9 As shown, the computing device cluster includes at least one computing device 100. The memory 106 in one or more computing devices 100 in the computing device cluster may store the same instructions for the execution method of the sensing task.

[0151] In some possible implementations, the memory 106 of one or more computing devices 100 in the computing device cluster may also store partial instructions for executing the training method of the multi-task model. In other words, the combination of one or more computing devices 100 can jointly execute instructions for executing the execution method of the perception task.

[0152] It should be noted that the memory 106 in different computing devices 100 in the computing device cluster can store different instructions, which are respectively used to execute part of the functions of the execution method of the perception task.

[0153] In some possible implementations, the memory 106 of one or more computing devices 100 in the computing device cluster may also store partial instructions for executing the execution method of the perception task. In other words, the combination of one or more computing devices 100 can jointly execute the instructions for executing the execution method of the perception task.

[0154] It should be noted that the memory 106 in different computing devices 100 in the computing device cluster can store different instructions for executing part of the functions of the training method of the multi-task model. That is, the instructions stored in the memory 106 in different computing devices 100 can implement the functions of one or more nodes in the task mining training module 501 and the shared storage module 502.

[0155] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Fig.10 A possible implementation is shown. Fig.10As shown, two computing devices 100A and 100B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 106 in the computing device 100A stores instructions for executing the task mining training module 501. At the same time, the memory 106 in the computing device 100B stores instructions for executing the functions of the shared storage module 502. It should be understood that Fig.10 The functions of the computing device 100A shown in FIG. 1 may also be completed by multiple computing devices 100. Similarly, the functions of the computing device 100B may also be completed by multiple computing devices 100.

[0156] The present disclosure also provides another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similar to that of Fig. 9 and Fig.10 The connection mode of the computing device cluster is different in that the memory 106 in one or more computing devices 100 in the computing device cluster may store the same instructions for executing the method for executing the sensing task.

[0157] In some possible implementations, the memory 106 of one or more computing devices 100 in the computing device cluster may also store partial instructions for executing the execution method of the perception task. In other words, the combination of one or more computing devices 100 can jointly execute the instructions for executing the execution method of the perception task.

[0158] It should be noted that the memory 106 in different computing devices 100 in the computing device cluster can store different instructions for executing part of the functions of the method for executing the perception task. That is, the instructions stored in the memory 106 in different computing devices 100 can implement the functions of one or more nodes in the task mining training module 501 and the shared storage module 502.

[0159] The present disclosure also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device is caused to perform a method for performing a perception task.

[0160] The present disclosure also provides a computer-readable storage medium. The computer-readable storage medium may be any available medium that can be stored by a computing device or a data storage device such as a data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to perform a method for executing a perception task.

[0161] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for executing a perception task, characterized in that: The method is applied to an edge-cloud collaborative system, the edge-cloud collaborative system includes an edge terminal and a cloud terminal, and the method includes: The cloud receives an initial perception task model and a plurality of perception data samples uploaded by a user, wherein the plurality of perception data samples belong to K-type task distributions, and K is an integer greater than 1; The cloud performs training on the initial perception task model based on the multiple perception data samples to obtain a multi-task model and L average task distribution representations, wherein the multi-task model includes K parameter separation model branches, and the K parameter separation model branches are respectively used to infer the perception data under the K types of task distributions; The cloud sends the multi-task model and the L average task distribution representations to the edge; The edge collects the perception data to be inferred; The edge end obtains a task distribution representation of the perception data to be inferred; The edge selects a parameter separation model branch to be used from the K parameter separation model branches based on the task distribution representation of the perception data to be inferred and the L average task distribution representations; The edge performs reasoning on the perception data to be inferred based on the parameter separation model branch to be used.

2. The method according to claim 1, characterized in that The cloud performs training on the initial perception task model based on the multiple perception data samples to obtain a multi-task model and L average task distribution representations, including: The cloud side trains the initial perception task model based on the multiple perception data samples to obtain a multi-task model, L average task distribution representations and K interpretable task distribution label groups, wherein the K interpretable task distribution label groups correspond one-to-one to the K parameter separation model branches.

3. The method according to claim 2, characterized in that The cloud performs training on the initial perception task model based on the multiple perception data samples to obtain a multi-task model, L average task distribution representations, and K interpretable task distribution label groups, including: The cloud obtains, for each perception data sample among the multiple perception data samples, a task distribution representation of the perception data sample and obtains an interpretable task distribution label of the perception data sample; The cloud clusters the multiple perception data samples into K sample sets based on the task distribution representation of the multiple perception data samples, wherein the K sample sets correspond to the K types of task distributions respectively, and each sample set includes at least one perception data sample; The cloud performs training on the last M layers in the initial perception task model for each sample set in the K sample sets based on the sample set to obtain a parameter separation model branch, uses the mean of the task distribution representations of each perception data sample in the sample set as the average task distribution representation corresponding to the trained parameter separation model branch, and uses the set of interpretable task distribution labels of each perception data sample in the sample set as the interpretable task distribution label corresponding to the trained parameter separation model branch; The cloud separates the first N layers of the initial perception task model and the K parameter separation model branches obtained through training to form a multi-task model, wherein the initial perception task model is composed of the first N layers and the last M layers, and in the initial perception task model, the first N layers are located before the last M layers.

4. The method according to claim 3, characterized in that The step of obtaining a task distribution representation of the perception data sample includes: The perception data samples are input into the first N layers to obtain intermediate feature data output by each of the first N layers, and based on the intermediate feature data output by each of the first N layers, a task distribution representation of the perception data samples is calculated.

5. The method according to claim 3 or 4, characterized in that: The obtaining of interpretable task distribution labels of the perception data samples includes: Acquire the perception data in the correspondence between the stored perception data and the task distribution explainable label, and for each acquired perception data, acquire the task distribution representation of the perception data; The similarity between the task distribution representation of the perception data sample and the task distribution representation of each perception data in the corresponding relationship is calculated, and the task distribution interpretable label corresponding to the perception data corresponding to the maximum similarity is used as the task distribution interpretable label corresponding to the perception data sample.

6. The method according to any one of claims 1 to 5, characterized in that The edge selects a parameter separation model branch to be used from the K parameter separation model branches based on the task distribution representation of the perception data to be inferred and the L average task distribution representations, including: The edge respectively calculates the similarity between the task distribution representation of the perception data to be inferred and the L average task distribution representations, and selects the parameter separation model branch corresponding to the average task distribution representation corresponding to the maximum similarity as the parameter separation model branch to be used.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: The cloud receives a model update request sent by a user, wherein the model update request carries a target average task distribution representation and a target perception data sample, wherein the target perception data sample is perception data under a target task distribution, and the target task distribution is a task distribution indicated by an interpretable label of a task distribution corresponding to the target average task distribution representation; The cloud end trains a parameter separation model branch corresponding to the target average task distribution representation in the multi-task model based on the target perception data sample.

8. A method for executing a perception task, characterized in that: The method is applied to a cloud end in an edge-cloud collaborative system, wherein the edge-cloud collaborative system also includes an edge end, and the method includes: Receiving an initial perception task model and a plurality of perception data samples uploaded by a user, wherein the plurality of perception data samples belong to K-type task distributions, where K is an integer greater than 1; Based on the multiple perception data samples, the initial perception task model is trained to obtain a multi-task model and L average task distribution representations, wherein the multi-task model includes K parameter separation model branches, and the K parameter separation model branches are respectively used to infer the perception data under the K types of task distributions; The multi-task model and the L average task distribution representations are sent to the edge.

9. A method for executing a perception task, characterized in that: The method is applied to an edge in an edge-cloud collaborative system, wherein the edge-cloud collaborative system also includes a cloud end, and the method includes: Receiving the multi-task model and the L average task distribution representations sent by the cloud, wherein L and K are integers greater than 1, and the multi-task model includes K parameter separation model branches, and the K parameter separation model branches are respectively used to infer the perception data under the K types of task distributions; Collect perception data to be inferred; Obtaining a task distribution representation of the perception data to be inferred; Based on the task distribution representation of the perception data to be inferred and the L average task distribution representations, selecting a parameter separation model branch to be used from the K parameter separation model branches; The to-be-inferred perception data is inferred based on the to-be-used parameter separation model branch.

10. A device for executing a perception task, characterized in that: The device is applied to a cloud end in an edge-cloud collaborative system, wherein the edge-cloud collaborative system also includes an edge end, and the device includes: A task mining training module is used to receive an initial perception task model and multiple perception data samples uploaded by a user, wherein the multiple perception data samples belong to K-type task distributions, and K is an integer greater than 1; based on the multiple perception data samples, the initial perception task model is trained to obtain a multi-task model and L average task distribution representations, wherein the multi-task model includes K parameter separation model branches, and the K parameter separation model branches are respectively used to infer the perception data under the K-type task distribution; A shared storage module is used to send the multi-task model and the L average task distribution representations to the edge.

11. A device for executing a perception task, characterized in that: The device is applied to an edge in an edge-cloud collaborative system, the edge-cloud collaborative system also includes a cloud end, and the device includes: A task distribution representation calculation module is used to receive the multi-task model and the L average task distribution representations sent by the cloud, wherein L and K are integers greater than 1, the multi-task model includes K parameter separation model branches, the K parameter separation model branches are respectively used to infer the perception data under the K types of task distribution, and the K parameter separation model branches correspond to the L average task distribution representations one by one; collect perception data to be inferred; obtain the task distribution representation of the perception data to be inferred; An adaptive model branch selection reasoning module is used to select a parameter separation model branch to be used from the K parameter separation model branches based on the task distribution representation of the perception data to be inferred and the L average task distribution representations; and to reason about the perception data to be inferred based on the parameter separation model branch to be used.

12. An edge, characterized in that: The edge includes a processor and a memory; The processor is used to execute instructions stored in the memory of the edge so that the edge executes the method for executing the perception task as described in claim 9.

13. A cloud, characterized in that: The cloud is a computer device cluster, the computer device cluster includes at least one computing device, each computing device includes a processor and a memory; The processor of at least one computing device is used to execute instructions stored in the memory of the device, so that the computing device cluster performs the method for executing the perception task as described in claim 8.

14. A computer program product comprising instructions, characterized in that When the instruction is executed by the edge, the edge executes the method for executing the perception task as described in claim 9.

15. A computer program product comprising instructions, characterized in that When the instruction is executed by the cloud, the cloud executes the method for executing the perception task as described in claim 8.

16. A computer-readable storage medium, characterized in that: It includes computer program instructions. When the computer program instructions are executed by the edge, the edge executes the method for executing the perception task as described in claim 9.

17. A computer-readable storage medium, characterized in that: It includes computer program instructions. When the computer program instructions are executed by the cloud, the cloud executes the method for executing the perception task as described in claim 8.

Images