Methods, apparatus, devices, media, and program products for executing task chains

Abstract

According to embodiments of this disclosure, a method, apparatus, device, medium, and program product for executing a task chain are provided. The method includes: in response to a first task in the task chain being executed by a first module, determining first evaluation information, the first evaluation information indicating uncertainty in the first module's execution of the first task; in response to the first evaluation information satisfying a first condition, constructing a plurality of perturbation samples corresponding to the input of the first task by processing the input of the first task; providing the plurality of perturbation samples to the first module to determine second evaluation information based on multiple processing results of the plurality of perturbation samples; in response to the second evaluation information satisfying a second condition, determining a second task from at least one prior task in the task chain that was executed before the first task; and triggering a second module to re-execute the second task. In this manner, this disclosure can suppress the cascading diffusion of uncertainty in the task chain, reduce the execution risk of the task chain, and ensure the accuracy of the execution results corresponding to the task chain.

Description

Technical Field

[0001] The exemplary embodiments disclosed herein relate generally to the field of computers, and more particularly to methods, apparatuses, devices, media, and program products for performing task chains. Background Technology

[0002] With the development of computer technology, various forms of electronic devices have greatly enriched people's daily lives. For example, people can use electronic devices to execute pipeline tasks in a task chain. However, when the output of a task has a certain degree of uncertainty but has not yet reached a certain threshold, if left unaddressed, the uncertainty of that task will often be propagated to subsequent tasks and amplified step by step, eventually leading to deviations in the execution result of the entire task chain or causing security risks. Therefore, how to perceive and handle the uncertainty of each task in real time during the execution of a task chain and make dynamic adjustments to ensure the safety and accuracy of the task chain execution is currently a key issue of concern. Summary of the Invention

[0003] In a first aspect of this disclosure, a method for executing a task chain is provided. The method includes: in response to a first task in the task chain being executed by a first module, determining first evaluation information, the first evaluation information indicating uncertainty in the first module's execution of the first task; in response to the first evaluation information satisfying a first condition, constructing a plurality of perturbation samples corresponding to the input by processing the input of the first task; providing the plurality of perturbation samples to the first module to determine second evaluation information based on multiple processing results of the plurality of perturbation samples; in response to the second evaluation information satisfying a second condition, determining a second task from at least one prior task in the task chain that was executed before the first task; and triggering a second module to re-execute the second task.

[0004] In a second aspect of this disclosure, an apparatus for executing a task chain is provided. The apparatus includes: a first determining module configured to determine first evaluation information in response to a first task in the task chain being executed by a first module, the first evaluation information indicating uncertainty in the first module's execution of the first task; a constructing module configured to construct a plurality of perturbation samples corresponding to the input of the first task by processing the input of the first task in response to the first evaluation information satisfying a first condition; a providing module configured to provide the plurality of perturbation samples to the first module to determine second evaluation information based on multiple processing results of the plurality of perturbation samples; a second determining module configured to determine a second task from at least one prior task in the task chain that was executed before the first task in response to the second evaluation information satisfying a second condition; and a first triggering module configured to trigger the second module to re-execute the second task.

[0005] In a third aspect of this disclosure, an electronic device is provided. The device includes at least one processor; and at least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor. When executed by the at least one processor, the instructions cause the device to perform the methods of the first or second aspect.

[0006] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores computer-executable instructions that can be executed by a processor to implement the method of the first aspect.

[0007] In a fifth aspect of this disclosure, a computer program product is provided. The computer program product includes computer-executable instructions that, when executed by a processor, implement the method according to a first aspect of this disclosure.

[0008] The embodiments disclosed herein can perform real-time uncertainty assessment and risk prediction for modules that perform tasks. If the risk prediction result is unacceptable, a review and verification of this module is initiated. If the risk result is still unacceptable after the review and verification, the process reverts to the preceding module that may have resulted in an unacceptable risk result to re-execute the preceding task. This can effectively suppress the cascading diffusion of uncertainty in the task chain, reduce the execution risk of the task chain, and ensure the accuracy of the execution results corresponding to the task chain.

[0009] It should be understood that the content described in this content section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0010] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0011] Figure 1 A schematic diagram of an example environment in which embodiments of the present disclosure can be implemented is shown;

[0012] Figure 2 A flowchart illustrating the execution of a task chain process according to some embodiments of the present disclosure is shown;

[0013] Figure 3 An example flowchart of an execution task chain according to some embodiments of the present disclosure is shown;

[0014] Figure 4 A schematic structural block diagram of an apparatus for performing a task chain according to certain embodiments of the present disclosure is shown;

[0015] Figure 5 A block diagram of an electronic device capable of implementing several embodiments of the present disclosure is shown. Detailed Implementation

[0016] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0017] It should be noted that the headings of any section / subsection provided herein are not limiting. Various embodiments are described throughout this document, and embodiments of any type may be included under any section / subsection. Furthermore, embodiments described in any section / subsection may be combined in any way with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0018] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below. The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0019] The embodiments of this disclosure may involve user data, data acquisition, and / or use. All of these aspects comply with applicable laws, regulations, and relevant provisions. In the embodiments of this disclosure, all data collection, acquisition, processing, manipulation, forwarding, and use are conducted with the user's knowledge and confirmation. Accordingly, in implementing the embodiments of this disclosure, the type, scope of use, and usage scenarios of any data or information that may be involved should be communicated to the user and their authorization obtained in accordance with relevant laws and regulations through appropriate means. The specific methods of notification and / or authorization may vary depending on the actual situation and application scenario, and the scope of this disclosure is not limited in this respect.

[0020] In this specification and the embodiments, any processing of personal information will be carried out only under the premise of legality (such as obtaining the consent of the personal information subject, or being necessary for the performance of a contract), and will only be carried out within the scope stipulated or agreed upon. A user's refusal to process personal information beyond what is necessary for basic functions will not affect the user's use of basic functions.

[0021] Furthermore, regarding the evaluation information involved in this disclosure (such as first evaluation information, second evaluation information, global evaluation information, etc.), if the corresponding evaluation information meets the corresponding conditions (such as the first condition, second condition, third condition, fourth condition, etc.), it can indicate that the current risk has reached a predetermined threshold. At this point, the risk is relatively high, and intervention is needed in the execution of the corresponding tasks in the task chain to prevent the risk from spreading. If the corresponding evaluation information does not meet the corresponding conditions, it can indicate that the current risk is within an acceptable range, allowing the task to continue. In addition, the specific descriptions of each condition involved in this disclosure are merely examples and do not constitute any limitation on this disclosure.

[0022] Embodiments of this disclosure propose a scheme for executing a task chain. According to this scheme, in response to a first task in the task chain being executed by a first module, first evaluation information can be determined, indicating the uncertainty of the first module executing the first task. Further, in response to the first evaluation information satisfying a first condition, multiple perturbation samples corresponding to the input of the first task can be constructed by processing the input. Further, multiple perturbation samples can be provided to the first module to determine second evaluation information based on multiple processing results of the multiple perturbation samples. Further, in response to the second evaluation information satisfying a second condition, a second task can be determined from at least one prior task in the task chain that was executed before the first task. Further, a second module can be triggered to re-execute the second task.

[0023] Based on this approach, the embodiments of this disclosure can perform real-time uncertainty assessment and risk prediction for the modules executing tasks. If the risk prediction result is unacceptable, a review and verification of this module is initiated. If the risk result is still unacceptable after the review and verification, the process reverts to the preceding module that may have resulted in an unacceptable risk result to re-execute the preceding task. This can effectively suppress the cascading diffusion of uncertainty in the task chain, reduce the execution risk of the task chain, and ensure the accuracy of the execution results corresponding to the task chain.

[0024] Example Environment

[0025] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. For example... Figure 1 As shown, example environment 100 may include task processing device 110.

[0026] In this example environment 100, the task processing device 110 can trigger the execution of each task in the task chain 120. In some embodiments, each task in the task chain 120 corresponds to any appropriate execution order, such as task 1, task 2, task 3, ... task N, etc., in sequential execution order. In addition, there may be correlations between the tasks, such as the output of a previous task being the input of one or more subsequent tasks, which will not be elaborated here.

[0027] In some embodiments, the task chain 120 can be a task chain associated with the task processing device 110 (for example, each task in the task chain 120 can be executed by the task processing device 110, or some tasks can be executed by the task processing device 110 while others are executed by other devices). The task chain 120 can also be a task chain associated with other devices (for example, each task in the task chain is executed by other devices), which will not be elaborated here.

[0028] In some embodiments, the task processing device 110 can be any type of mobile terminal, fixed terminal, or portable terminal with a display device, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, handheld computers, portable gaming terminals, VR / AR devices, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio receivers, e-book devices, gaming devices, or any combination thereof, including accessories and peripherals of these devices or any combination thereof. In some embodiments, the task processing device 110 can also support any type of interface for the target user (such as "wearable" circuitry). The task processing device 110 can also be a robot, etc.

[0029] Task processing device 110 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms. Task processing device 110 may include, for example, computing systems / servers such as mainframes, edge computing nodes, computing devices in cloud environments, etc.

[0030] It should be understood that the structure and function of the various elements in environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure.

[0031] The following description will continue with reference to the accompanying drawings, which will provide some exemplary embodiments of this disclosure.

[0032] Example process

[0033] Figure 2 A flowchart of a task chain execution process 200 according to some embodiments of the present disclosure is shown. Process 200 can be implemented at task processing device 110. Reference is made below. Figure 1 Describe the process 200.

[0034] In block 210, in response to the execution of the first task in the task chain by the first module, the task processing device 110 determines first evaluation information, which indicates the uncertainty of the first module's execution of the first task.

[0035] In some embodiments, a task chain can be a set of tasks corresponding to multiple tasks that need to be executed to achieve a predetermined goal, and each task in this task set corresponds to a predetermined execution order. For example, the task set includes task A, task B, task C, and task D, and these tasks are expected to be executed sequentially in the order of task A, task C, task B, and task D.

[0036] In some embodiments, for at least two of these multiple tasks, the execution order of these at least two tasks may be the same (i.e., these at least two tasks are executed in parallel without any sequential execution relationship), which will not be elaborated here.

[0037] In some embodiments, a task chain may include multiple tasks corresponding to any suitable type of pipeline. As an example, a task chain may include multiple tasks corresponding to a robotic pipeline. As an example, a task chain may include multiple tasks corresponding to a pipeline for daily services (such as ordering takeout, online shopping, etc.).

[0038] In some embodiments, the first task may be a non-first task in the task chain, that is, at least one other task in the task chain has been completed before the first task is executed.

[0039] In some embodiments, the type corresponding to the first task can be any suitable type. For example, taking a task chain that includes multiple tasks corresponding to a robot assembly line as an example, the first task can be an object detection task, a semantic segmentation task, an image processing task, a perception task, a recommendation task, a decision planning task, etc. As another example, taking a task chain that includes multiple tasks corresponding to a life service assembly line as an example, the first task can be a shopping task, an order placement task, etc.

[0040] In some embodiments, the first module performing the first task can be any suitable module, such as a module based on a machine learning model. Of course, the first module can also be a non-learning, ordinary module, such as a module based on a predetermined algorithm or predetermined rules. In some embodiments, the first module can be a software algorithm unit or a hardware computing unit, etc.

[0041] In some embodiments, uncertainty indicates the degree of lack of reliability and stability of the processing result output by the first module for a given current input. Taking a machine learning model as an example, this uncertainty can be related to the model's confidence in the output result; for example, the lower the confidence, the greater the uncertainty.

[0042] In order to facilitate the measurement of the risk level of task execution, in some embodiments, the task processing device 110 can determine first evaluation information, which can be a quantitative numerical indicator or a level indicator, used to characterize the degree or level of uncertainty of the first module performing the first task.

[0043] The process for determining the first evaluation information is explained below:

[0044] In some embodiments, the task processing device 110 may determine an uncertainty measure corresponding to the first task in response to the execution of the first task. The uncertainty measure is a quantitative indicator of the degree of lack of reliability and stability of the processing result output by the first module for a given current input.

[0045] In some embodiments, for a first module that can output a probability distribution or can transform the output into a probability distribution, the task processing device 110 can determine an uncertainty measure based on the probability distribution.

[0046] Specifically, for the first module that can output a probability distribution, uncertainty measures may include, but are not limited to, prediction entropy, normalized entropy, alternative risk indicators (such as those determined based on the maximum class probability or the Top-2 interval), etc.

[0047] As an example, for the discrete probability distribution output by the first module, this disclosure can determine the prediction entropy based on the following formula:

[0048] ；

[0049] in To predict entropy, It is a discrete probability distribution, which indicates the probability value of determining whether the current input of the module belongs to the first predefined category. The probability value of determining whether the current input of the module belongs to the second predefined category. The probability value of determining whether the current input of the module belongs to the i-th predefined category. Wait, C is the total number of predefined categories.

[0050] To eliminate the influence of the number of categories C on the range of entropy values ​​and to facilitate comparisons across modules, as another example, for the discrete probability distribution output by the first module, this disclosure can determine the normalized entropy based on the following formula:

[0051] ；

[0052] ；

[0053] in, To predict entropy, This is the normalized entropy, and the value is normalized to the interval [0, 1]. It is a discrete probability distribution, which indicates the probability value of determining whether the current input of the module belongs to the first predefined category. The probability value of determining whether the current input of the module belongs to the second predefined category. The probability value of determining whether the current input of the module belongs to the i-th predefined category. Wait, C is the total number of predefined categories. It is the theoretical maximum entropy.

[0054] It should be noted that the higher the entropy value corresponding to the prediction entropy or normalized entropy, the greater the uncertainty in the module's task execution.

[0055] As another example, for the discrete probability distribution output by the first module, this disclosure can determine the alternative risk indicator r based on the following formula:

[0056] ;

[0057] Where r is an example of an alternative risk indicator determined based on the maximum class probability. Discrete probability distribution The probability value that determines it belongs to the i-th predefined category. It represents The highest probability value among them.

[0058] As another example, for the discrete probability distribution output by the first module, this disclosure can determine the alternative risk indicator r based on the following formula:

[0059] ；

[0060] Where r is an example of an alternative risk indicator determined based on the maximum class probability. Discrete probability distribution The highest probability value in the middle. Discrete probability distribution The second highest probability value.

[0061] It should be noted that, The larger the value, the greater the uncertainty in the module's task execution.

[0062] Specifically, for cases where the output of the first module can be converted into a probability distribution, uncertainty measures can include, but are not limited to, pixel-level entropy, spatial entropy, differential entropy, channel entropy, etc.

[0063] As an example, considering the first module's handling of classification tasks, the output layer of the first module typically includes a normalization function (such as the Softmax function), which is used to transform the original Logits vector into a probability distribution. Specifically, the components of the probability distribution p can be determined based on the following formula:

[0064] ；

[0065] in, Discrete probability distribution The probability value that determines it belongs to the i-th predefined category. It is the j-th component in the original Logits vector. Let be the i-th component in the original Logits vector, and C be the total number of components in the original Logits vector, where the Logits vector is... .

[0066] It should be noted that the converted value at this time... This refers to the category probability value calculated for each detection box or each pixel location. Furthermore, the task processing device 110 can determine the prediction entropy or normalized entropy from the transformed probability distribution; the specific determination method has already been described and will not be repeated here.

[0067] As another example, taking the first module (such as the segmentation model) as an example of handling semantic segmentation / instance segmentation tasks, the first module can output a three-dimensional tensor after processing by the normalization (softmax) layer. Its size is [Height, Width, C], where Height is the height of the image (number of pixel rows), Width is the width of the image (number of pixel columns), and C is the total number of semantic categories (e.g., "background", "person", "vehicle", "road", etc.).

[0068] Specifically, the task processing device 110 can extract the class probability vector corresponding to the pixel position (x, y). ,in For each pixel position (x, y) in the image, a one-dimensional vector of length C along the channel dimension is extracted from the three-dimensional tensor P. =[P[x,y,1],P[x,y,2],...,P[x,y,C]]. Furthermore, the task processing device 110 can process each probability value in the vector. ,calculate ,in for The i-th element in the dataset. Furthermore, the task processing device 110 can process all categories of... The results are summed. Furthermore, the task processing device 110 can take the negative value of the summation result to obtain the entropy. ,in -Σ( ), where the summation is performed from 1 to C, It is a vector The i-th element in the array, where log represents the natural logarithm. The pixel position (x, y) corresponds to... It represents a measure of classification uncertainty.

[0069] Due to the above This represents a pixel-level uncertainty measure. To further obtain instance or image-level risk scores, the task processing device 110 can aggregate pixel-level entropy. Specific aggregation methods may include, but are not limited to, calculating the average pixel entropy within the target region, the upper quantile (such as the 90th percentile), or the proportion of high-entropy pixels, etc., which will not be elaborated here.

[0070] As an example, taking the task processed by the first module as an object detection task, the output of the first module is the detected bounding box, class probability, and existence confidence. At this time, the uncertainty measure can be determined based on the bounding box, class probability, and existence confidence output by the first module based on the following process.

[0071] Specifically, the task processing device 110 can determine the probability vector of each bounding box belonging to each category. This probability vector can also be called the category probability distribution, where the category probability distribution of bounding box j can be expressed as: Furthermore, the task processing device 110 can determine... Information entropy Furthermore, the task processing device 110 can determine the uncertainty measure based on the following formula:

[0072] ;

[0073] in Let j be the uncertainty measure corresponding to the bounding box. Let be the class probability distribution of bounding box j. Let j be the information entropy corresponding to the border. Let j be the existence confidence level corresponding to the bounding box. These are the pre-defined parameters.

[0074] Due to the above This characterizes the uncertainty measure at the bounding box level. To further obtain the image-level uncertainty measure, the task processing device 110 can perform bounding box-level... Aggregation is performed. Specific aggregation methods may include, but are not limited to: retrieving all ultimately retained detection boxes. The maximum value or average value, etc., will not be elaborated here.

[0075] As an example, taking the task processed by the first module as a heatmap-based task, the output of the first module is a heatmap. This heatmap can be a two-dimensional matrix, and the value H_k(u,v) of each position (u,v) in the two-dimensional matrix represents the "response intensity" or "probability" that the first module believes the key point is located at that pixel position. Further, the task processing device 110 can normalize the heatmap to obtain a normalized heatmap. Further, the task processing device 110 can view the normalized heatmap as a probability distribution at spatial positions (u,v) and determine its corresponding information entropy (i.e., uncertainty measure) based on the probability distribution. This information entropy can also be called the spatial entropy of the heatmap, which measures the concentration of the predicted probability corresponding to the key point in the entire image space.

[0076] It should be noted that the higher the spatial entropy of the heatmap, the greater the uncertainty in the execution of the task by the first module.

[0077] As another example, taking the first module as an example of handling continuous regression tasks (such as depth estimation, bounding box coordinate prediction, etc.), the output of the first module is usually a continuous value.

[0078] For example, if the first module outputs the mean μ and variance σ², then this disclosure can determine the uncertainty measure based on the following formula, where the uncertainty measure is the differential entropy:

[0079] ；

[0080] Where h is the differential entropy, Let Variance be the variance.

[0081] It should be noted that the standard deviation σ can be calibrated (e.g., through standard deviation calibration) before using the above formula to ensure that it reflects the true predictive fluctuations. Furthermore, a higher differential entropy indicates greater uncertainty in the first module's task execution. Since h has a monotonically related relationship with the standard deviation σ, in some other embodiments, the calibrated standard deviation σ or variance σ² can be directly used as an uncertainty measure, employing the normalization strategy described below for "unbounded positive uncertainty such as variance".

[0082] For example, taking the first module's processing of full-image regression tasks (such as depth estimation, optical flow prediction, etc.) as an example, the task processing device 110 can normalize the final layer feature map of the first module (model) into a probability distribution along the channel dimension. Furthermore, the task processing device 110 can calculate the channel entropy at each spatial location based on the probability distribution, that is, the information entropy of the probability distribution along the channel dimension at that spatial location.

[0083] Since the channel entropy described above represents an uncertainty measure at the spatial location level, in order to further obtain an image-level uncertainty measure, the task processing device 110 can aggregate the channel entropy corresponding to each spatial location based on a predetermined aggregation method. The predetermined aggregation method may include, but is not limited to, averaging or weighted summation, etc.

[0084] As another example, taking the bounding boxes in the multiple target detections output by the first module as an example, the task processing device 110 can discretize the continuous parameters into a histogram distribution based on multiple prediction candidates (such as center coordinates, width and height) for the same target. Furthermore, the task processing device 110 can determine the entropy (uncertainty measure) of the histogram distribution based on the histogram distribution.

[0085] It should be noted that the lower the entropy of the histogram distribution, the more consistent the multiple prediction candidates are, indicating that the uncertainty of the first module's task execution is smaller.

[0086] As another example, taking the first module's handling of visual language models and text generation tasks, the generation process can be viewed as a token-by-token classification. Specifically, for each time step t, the first module can output a probability distribution in the vocabulary. Furthermore, the task processing device 110 can be based on a probability distribution. Calculate the entropy corresponding to the token. Furthermore, the task processing device 110 can determine the normalized entropy corresponding to the token based on the entropy corresponding to the token. ,in Let V be the entropy, and V be the vocabulary size. It represents the maximum entropy.

[0087] To assess the uncertainty of the entire generated sequence, the normalized entropy at the token level can be aggregated to obtain a measure of the uncertainty of the entire sequence. Specific aggregation methods can include, but are not limited to, averaging, finding the maximum value, etc.

[0088] To ensure the reliability of the uncertainty measure determined based on the output of the first module, the task processing device 110 can perform a predetermined calibration on the output of the model and determine the uncertainty measure corresponding to the module based on the calibrated output.

[0089] For example, for the module that outputs the Logits vector, the task processing device 110 can learn a scalar temperature parameter. Furthermore, the task processing device 110 can optimize the probability p (e.g., by minimizing the negative log-likelihood) to make the calibrated probability... The confidence and accuracy are aligned, where the Logits vector is... T is a scalar temperature parameter.

[0090] It should be noted that when This will make the probability distribution smoother (increase entropy). This makes the probability distribution sharper (entropy decreases).

[0091] For example, take the average output of the first module. and variance For example, task processing device 110 can learn a scalar. The output variance is calibrated to match the actual distribution of the prediction error.

[0092] Specifically, this disclosure allows for the calibration of the standard deviation based on the following calibration formula:

[0093] ;

[0094] in, As a scalar, The standard deviation before calibration. This is the calibrated standard deviation.

[0095] In some embodiments, the scalar s can be determined based on the following formula:

[0096] ；

[0097] Where s is a scalar, The total number of samples, For the true value, This is the mean of the original output of the first module. Let be the variance of the original output of the first module, and let i represent the i-th sample.

[0098] Furthermore, the task processing device 110 can perform normalization processing on the uncertainty measure to determine the risk value corresponding to the first task. At this time, the risk value obtained after normalization corresponds to the range [0,1].

[0099] Specifically, the task processing device 110 can determine a normalization strategy that matches the task type from multiple preset normalization strategies based on the task type of the first task.

[0100] In some embodiments, if the task type of the first task indicates that the output of the first task after execution by the first module can determine an uncertainty measure of entropy (such as normalized entropy, spatial entropy, prediction entropy, etc.), then the normalization strategy determined based on this task can be normalized based on the following formula:

[0101] ；

[0102] Where A is the risk value. For the original entropy, This represents the theoretical maximum value that this entropy measure can achieve under predetermined conditions, such as for text generation tasks. For image classification, Where C is the predetermined number of categories and V is the size of the vocabulary. In other embodiments, for tasks with continuously distributed outputs, the range of uncertainty measures (such as differential entropy) may include negative numbers. In this case, the differential entropy can be mapped to the [0,1] interval using any suitable normalization function, such as the Logistic function (Sigmoid), or any other normalization function.

[0103] In other embodiments, taking the example that the task type of the first task indicates that the output after the first task is executed by the first module is an unbounded positive number such as variance, then the normalization strategy determined based on this task can be normalized based on the following formula:

[0104] ;

[0105] Where A is the risk value. λ is the standard deviation, and λ is the task-related scaling factor.

[0106] Furthermore, the task processing device 110 can perform normalization processing on the uncertainty measure based on a normalization strategy to determine the risk value corresponding to the first task. Further, the task processing device 110 can determine first evaluation information based on the risk value. As an example, the task processing device 110 can directly determine the risk value as the first evaluation information. As another example, the task processing device 110 can determine the target risk level corresponding to the risk value based on the risk value and a predetermined set of risk areas corresponding to risk levels, so as to determine the target risk level as the first evaluation information.

[0107] In box 220, in response to the first evaluation information satisfying the first condition, the task processing device 110 processes the input of the first task to construct multiple perturbation samples corresponding to the input.

[0108] In some embodiments, the first condition can be any suitable condition, such as a risk value greater than a predetermined threshold A, which can be set as needed. As an example, the threshold A can be determined on the validation set based on the target false positive rate or false negative rate using the receiver operating characteristic (ROC) curve or the precision-recall (PR) curve, which will not be elaborated here.

[0109] In some embodiments, when the first evaluation information meets the first condition, it indicates that the uncertainty of the first module executing the first task is high, and the execution of the task chain carries a certain risk. To accurately determine the uncertainty of the first module executing the first task, the task processing device 110 can process the input of the first task to construct multiple perturbation samples corresponding to the input. In some embodiments, these multiple perturbation samples are configured to be provided to the first module so that the first module can redetermine the uncertainty based on multiple processing results obtained from processing these multiple perturbation samples.

[0110] In some embodiments, the number of these perturbation samples can be set as needed, and in order not to change the semantics of the input, the degree of perturbation can be less than a threshold to ensure that the perturbation samples are still within the range that the first module can process.

[0111] Taking an input that includes image content as an example, in some embodiments, the task processing device 110 can perform geometric transformations and / or scaling transformations on the image content in the input to construct perturbation samples. In some embodiments, geometric transformations may include, but are not limited to, flip transformations, rotation transformations, etc. In some embodiments, scaling transformations may include, but are not limited to, scaling transformations and cropping transformations, etc.

[0112] For example, the task processing device 110 can horizontally flip the image content or rotate the image content by a preset angle (such as 90 degrees). Another example is that the task processing device 110 can scale the image content to half its original size.

[0113] In other embodiments, the task processing device 110 may also add color perturbations to the image content in the input to construct perturbation samples.

[0114] Taking the first module as an example, which is based on a machine learning model (such as a language model or a multimodal model), in other embodiments, the task processing device 110 can rewrite the prompts in the input to construct perturbation samples. As an example, the task processing device 110 can paraphrase the prompts.

[0115] In other embodiments, the task processing device 110 may also construct perturbation samples by rewriting prompts in the input, performing geometric transformations and / or scaling transformations on the image content in the input, and adding color perturbations to the image content in the input.

[0116] In other embodiments, the task processing device 110 can also fine-tune the decoding temperature to construct perturbation parameters. That is, the task processing device 110 can control text generation by adjusting the temperature hyperparameter during the inference (decoding) phase of the text generation model. Specifically, the task processing device 110 can generate multiple different outputs for the same input by generating text multiple times at different decoding temperatures. Furthermore, the task processing device 110 can redetermine the uncertainty of the first module performing the first task based on the consistency among these multiple different outputs and / or the prediction distribution derived therefrom.

[0117] Furthermore, in response to the second evaluation information not meeting the second condition, the task processing device 110 can determine the output result of the first task based on multiple processing results. This output result is the final processing result corresponding to the first task. As an example, the task processing device 110 can aggregate these multiple processing results based on a predetermined aggregation method to determine the output result of the first task.

[0118] It should be noted that when the tasks processed by the module correspond to different task types, the aggregation methods for these multiple processing results may also be different. In some embodiments, the aggregation methods may include, but are not limited to: taking the class with the highest probability after averaging or weighted averaging multiple output probability distributions, performing majority voting on multiple class prediction results, averaging, medianing or weighted statistics on multiple regression results, and selecting the final sequence based on the consistency of multiple generated sequences in sequence generation tasks, etc.

[0119] For example, if the first module is configured to perform an object detection task, and the first module detects that the object in the image is a puppy, and the first evaluation information at this time satisfies the first condition, then the task processing device 110 can process this image to form three perturbation images. Further, the task processing device 110 can provide these three perturbation images to the first module to generate three processing results corresponding to these three perturbation images. If the first processing result determined based on the first perturbation image indicates that the object in the image is a kitten, the second processing result determined based on the second perturbation image indicates that the object in the image is a puppy, and the third processing result determined based on the third perturbation image indicates that the object in the image is a kitten, and the second evaluation information determined based on these three processing results does not satisfy the second condition, then by aggregating these three processing results, the final processing result of the first module performing the object detection task can be determined as the object in the image being a kitten.

[0120] When the second evaluation information does not meet the second condition, it indicates that the reliability of the first module in executing the task is relatively high, and the risk of the first module in processing the task is also relatively low. Therefore, the task processing device 110 can also, in response to the second evaluation information not meeting the second condition, determine the processing result obtained from the first execution of the first task as the output result of the first task. The evaluation information corresponding to the first execution of the first task is the first evaluation information.

[0121] In some embodiments, the task processing device 110 may trigger a third module to execute a third task in the task chain in response to the first evaluation information not meeting a first condition. For example, when the risk value is less than or equal to a predetermined threshold A, the third module is triggered to execute the third task in the task chain. The third task is a task to be executed after the first task in the task chain.

[0122] In some embodiments, the third module performing the third task can be any suitable module, such as a module based on a machine learning model. Of course, the third module can also be a non-learning, ordinary module, such as a module based on a predetermined algorithm or predetermined rules. In some embodiments, the third module can be a software algorithm unit or a hardware computing unit, etc.

[0123] Since the uncertainty of a single module may not be very large, but there are many tasks in the task chain, the accumulation of uncertainty of each task may increase the overall execution risk of the task chain and affect the accuracy of task execution. Therefore, the task processing device 110 can also update the global evaluation information associated with the task chain based on the first evaluation information.

[0124] As an example, task processing device 110 can determine an initial risk value indicated by global evaluation information. The initial risk is associated with the evaluation information corresponding to all tasks executed prior to the first task. Further, task processing device 110 can apply a predetermined forgetting coefficient to attenuate the initial risk value to determine a first risk value. For example, task processing device 110 can determine the first risk value by multiplying the forgetting coefficient by the initial risk value. The forgetting coefficient can also be called a forgetting factor. The forgetting coefficient can be any suitable value less than 1, which can be set as needed. Further, task processing device 110 can determine an updated risk value for the global evaluation information based on the first risk value and a second risk value determined based on the first and second evaluation information.

[0125] In some embodiments, the second risk value may be the same as the risk value indicated by the second evaluation information, or it may be the same as the risk value indicated by the first evaluation information. Alternatively, it may be determined by multiplying the risk value indicated by the first or second evaluation information by a predetermined weight, where the predetermined weight may be a weight set for the task (task weight or stage weight). In other embodiments, the second risk value may be determined based on the risk values ​​indicated by the first and second evaluation information, for example, by the mean, weighted sum, or other methods of the risk values ​​indicated by these two evaluation information.

[0126] As an example, the task processing device 110 can determine the updated risk value of the global evaluation information based on the sum of a first risk value and a second risk value determined based on the first evaluation information and / or the second evaluation information.

[0127] For example, task processing device 110 can determine the updated risk value based on the following formula:

[0128] ;

[0129] in, This is the updated risk value. The first risk value, The second risk value, Forgetting coefficient As the weight of the process, As the initial risk value, The risk value is used to determine the first evaluation information.

[0130] As another example, the task processing device 110 can determine the update risk value of the global evaluation information based on the weight sum of the first risk value and the second risk value, wherein the weight values ​​corresponding to the first risk value and the second risk value can be set according to requirements. As another example, the task processing device 110 can also determine the update risk value of the global evaluation information based on the product of the first risk value and the second risk value or other monotonic combinations, which will not be elaborated here.

[0131] Furthermore, the task processing device 110 can terminate the execution of the task chain in response to the global evaluation information meeting the third condition, thereby ensuring system security. The third condition can be any suitable condition, such as being greater than a predetermined threshold B, which can be set as needed. In some embodiments, the predetermined threshold B is greater than a predetermined threshold A.

[0132] In some embodiments, if the remaining computing power of the task processing device 110 corresponding to the task chain is less than a certain threshold and the execution time is greater than a certain threshold, the task chain execution can also be terminated to ensure the security of the task chain execution.

[0133] In some embodiments, the task processing device 110 may trigger the fourth module to re-execute the fourth task in the task chain in response to the global evaluation information satisfying the fourth condition. The fourth task is executed before the second task. The fourth condition can be any suitable condition, such as being greater than a predetermined threshold C, which can be set as needed.

[0134] In some embodiments, the fourth task can be any suitable task, such as a task with a risk value greater than a predetermined threshold. This indicates that the risk indicated by the global evaluation information is higher, and directly reverting to the previous one or a few (to a limited extent) will not significantly reduce the global risk. Therefore, a task that is earlier in the task chain can be re-executed to reduce the risk of uncertainty propagating through multiple levels in the task chain.

[0135] In box 230, task processing device 110 provides multiple perturbation samples to the first module to determine second evaluation information based on multiple processing results of the multiple perturbation samples.

[0136] In some embodiments, the task processing device 110 may determine second evaluation information based on the consistency of the plurality of processing results. As an example, the task processing device 110 may determine the second evaluation information by determining the similarity between the plurality of processing results.

[0137] In other embodiments, the task processing device 110 may acquire multiple processing results generated by the first module processing multiple perturbation samples. Further, the task processing device 110 may determine second evaluation information based on the degree of uncertainty corresponding to the multiple processing results and / or the differences between the multiple processing results.

[0138] In some embodiments, the task processing device 110 can determine multiple output probability distributions corresponding to multiple processing results. Further, the task processing device 110 can determine the average distribution of the multiple output probability distributions. Further, the task processing device 110 can determine the degree of uncertainty corresponding to the multiple processing results based on the normalized entropy of the average distribution.

[0139] For example, task processing device 110 can determine the degree of uncertainty based on the following formula:

[0140] ;

[0141] ;

[0142] in, The average distribution of multiple processing results. This module outputs the probability distribution after inputting a perturbation sample k into the first module. K is the total number of these processing results (or multiple perturbation samples), and C is the total number of categories. The entropy value corresponding to the average distribution. The degree of uncertainty (normalized entropy). It is the theoretical maximum entropy.

[0143] For example, task processing device 110 can determine the differences between multiple processing results based on the following formula:

[0144] ;

[0145] in, For the differences between multiple processing results, This is the probability distribution output by this module after the perturbation sample k is input into the first module. It is the first The probability distribution of predictions corresponding to each interference sample Shannon entropy, K represents the entropy value corresponding to the average distribution, and K is the total number of these multiple processing results (or multiple perturbation samples).

[0146] In some embodiments, the task processing device 110 may determine that the second evaluation information meets the second condition in response to the uncertainty level reaching a first threshold. The first threshold can be set as needed.

[0147] In other embodiments, the task processing device 110 may determine that the second evaluation information meets the second condition in response to a difference reaching a second threshold. The second threshold may also be set as needed.

[0148] In other embodiments, the task processing device 110 may also determine that the second evaluation information satisfies the second condition in response to the uncertainty level reaching a first threshold and the difference reaching a second threshold.

[0149] In box 240, in response to the second evaluation information satisfying the second condition, the task processing device 110 determines the second task from at least one prior task in the task chain that was executed before the first task.

[0150] In some embodiments, the second condition can be any suitable condition. For example, it could be that the risk value indicated by the second evaluation information is greater than a predetermined threshold, etc.

[0151] In some embodiments, the task processing device 110 may determine the latest executed task among at least at least at least prior tasks that were executed before the first task as the second task. That is, the task processing device 110 may determine the task preceding the first task among at least at least prior tasks as the second task. This preceding task is a task in the task chain that is adjacent to the first task and whose execution order precedes the first task.

[0152] In other embodiments, the task processing device 110 may determine a second task from at least one prior task based on evaluation information corresponding to at least one prior task. As an example, the task processing device 110 may determine the prior task with the lowest evaluation information as the second task. As another example, the task processing device 110 may determine the prior task whose evaluation information meets predetermined requirements as the second task.

[0153] In other embodiments, the task processing device 110 may trigger the third module to execute a third task in the task chain in response to the second evaluation information not meeting the second condition. For example, when the risk value is less than or equal to a predetermined threshold D, the third module is triggered to execute the third task in the task chain. The third task is the task to be executed after the first task in the task chain.

[0154] In some embodiments, the third module performing the third task can be any suitable module, such as a module based on a machine learning model. Of course, the third module can also be a non-learning, ordinary module, such as a module based on a predetermined algorithm or predetermined rules. In some embodiments, the third module can be a software algorithm unit or a hardware computing unit, etc.

[0155] Since the uncertainty of a single module may not be significant, but there are many tasks in a task chain, the accumulated uncertainty of each task may increase the overall execution risk of the task chain and affect the accuracy of task execution. Therefore, as an example, the task processing device 110 can also update the global evaluation information associated with the task chain based on the first evaluation information. As another example, the task processing device 110 can also update the global evaluation information associated with the task chain based on the second evaluation information. As yet another example, the task processing device 110 can also update the global evaluation information associated with the task chain based on both the first and second evaluation information. For example, the global evaluation information associated with the task chain can be updated based on the average or weighted sum of the risk values ​​indicated by the first and second evaluation information.

[0156] It should be noted that the process of determining global evaluation information and determining how to execute tasks in the task chain and whether to stop executing the task chain based on global evaluation information is the same as the process corresponding to the first evaluation information not meeting the first condition, and will not be elaborated here.

[0157] In box 250, task processing device 110 triggers the second module to re-execute the second task.

[0158] In some embodiments, the task processing device 110 may determine third evaluation information in response to the completion of the second task, the third evaluation information indicating the uncertainty of the second module's execution of the second task. Further, the task processing device 110 may still determine whether to execute a subsequent task based on whether the third evaluation information satisfies the first condition. For example, if the second task is a task adjacent to and preceding the first task in a task chain, then if the third evaluation information does not satisfy the first condition, the first module may be triggered to execute the first task again until the first evaluation information determined during the execution of the first task no longer satisfies the first condition, at which point the subsequent task can be executed.

[0159] Figure 3 Example flowchart 300 of an execution task chain according to some embodiments of the present disclosure is shown, now for... Figure 3 The following explanation will be provided for clarity. For ease of description, the example below will be the robot (task processing device 110) executing various tasks within the task chain.

[0160] In box 301, the robot can perform task m.

[0161] In some embodiments, task m can be the currently executing task in a task chain, and this task chain also contains other tasks such as task m-1 that have been completed before task m. Each task in the task chain can be executed by multiple modules corresponding to the robot, and each task has a corresponding module.

[0162] In some embodiments, the objective of this task chain can be that the robot can grab a specified "metal wrench" from a variety of items randomly stacked on a workbench and put it into a collection box.

[0163] At this point, the modules in the robot used to perform the various tasks included in the task chain may include, but are not limited to: scene analysis and target selection module, target detection module, instance segmentation module, motion planning and control module, etc.

[0164] Specifically, the scene analysis and target selection module can be configured to analyze the scene based on received natural language commands (such as a voice message about "grabbing a metal wrench") and output a pixel coordinate point pointing to the preferred grasping object. The object detection module can be configured to perform object detection in the corresponding surrounding area centered on the pixel coordinates to determine the bounding box of the grasping object. The instance segmentation module can be configured to output a pixel-level mask of the grasping object based on the bounding box, used for grasping pose calculation. The motion planning and control module can be configured to plan the grasping trajectory and execute the grasping based on information such as the pixel-level mask of the grasping object.

[0165] For ease of description, we will use task m as an example of the task executed by the target detection module.

[0166] In box 302, the robot can extract and calibrate the uncertainty measure for performing task m. .

[0167] As an example, the robot can obtain the classification distribution corresponding to the bounding boxes output by the object detection module. Furthermore, the robot can determine the entropy value based on the classification distribution; this entropy value is also a measure of uncertainty. .

[0168] In box 303, the robot can determine the risk value. .

[0169] In some embodiments, This risk value can be any appropriate normalization function. A measure of uncertainty The normalization process ensures that the risks of all tasks can be easily aggregated on the same scale.

[0170] In some embodiments, uncertainty measurement If the entropy value is determined based on the classification distribution, then the risk value can be the normalized entropy determined based on this entropy value.

[0171] In box 304, the robot can determine whether... And global risk value This is used to determine whether the first-level gate control has been passed.

[0172] In some embodiments, a and b are thresholds set according to requirements.

[0173] As an example, the robot can respond to And global risk value Once the first-level gate is cleared, the robot can then perform the operation in box 305.

[0174] As another example, the robot can respond to or global risk value The system determines that the first-level gate is blocked. Further, the robot can perform the operation in box 306.

[0175] In box 305, the robot updates the global risk value B.

[0176] As an example, task processing device 110 can determine the updated global risk value based on the following formula:

[0177] ;

[0178] in, This is the updated global risk value. The global risk value before the update. For the task The corresponding risk value, Forgetting coefficient The preset weights are for task m.

[0179] In box 306, the robot can re-execute task m.

[0180] In some embodiments, the robot can construct multiple interference samples based on the input of the module executing task m. Furthermore, the robot can determine multiple execution results by providing these multiple interference samples to the module executing task m.

[0181] As an example, the robot can construct multiple interfering samples based on the input (i.e., an image indicating pixel coordinates) to the object detection module performing task m. For example, it can scale and flip the original image at multiple scales to construct five interfering samples. Furthermore, the robot can provide these five interfering samples to the object detection module to determine five execution results corresponding to these five interfering samples.

[0182] In box 307, the robot can determine the risk value of re-judgment. With Disagreement .

[0183] In some embodiments, the robot can determine the degree of divergence based on the differences between these multiple processing results. .

[0184] In some embodiments, the robot can determine multiple output probability distributions corresponding to multiple processing results. Further, the robot can determine the average distribution of the multiple output probability distributions. Further, the robot can determine the re-judgment risk value corresponding to the multiple processing results based on the normalized entropy of the average distribution. .

[0185] In box 308, the robot can determine whether... and This is used to determine whether the secondary gate control is passed.

[0186] As an example, a robot can respond to a determination and The secondary gate control is confirmed to be in place. Further, the robot can perform the operations in box 305.

[0187] As another example, a robot can respond to a determination or The secondary gate control was determined to have failed. Further, the robot can execute the operation in box 309.

[0188] In some embodiments, a and f can be the same threshold or different thresholds, which can be set according to requirements.

[0189] In box 309, the robot can determine the upstream backoff task l.

[0190] In some embodiments, task l can be any appropriate task in the task chain that has already been executed and whose execution order precedes task m.

[0191] As an example, task l can be the previous task of task m.

[0192] As another example, task l can be the task with the highest risk value among all candidate tasks that have been completed in the task chain and whose execution order is before task m, and so on.

[0193] In box 310, the robot performs a rollback and recalculation operation.

[0194] In some embodiments, the robot can revert to task l and re-execute task l.

[0195] In addition, the risk value of task l can be re-determined based on the execution results obtained by re-executing task l.

[0196] At this point, the global risk value only retains the portion corresponding to the previous task of task l, while the portion corresponding to subsequent tasks after task l may not be retained in the global risk value. After task l is executed, the robot can update the global risk value based on the risk value corresponding to task l.

[0197] To facilitate backtracking, this disclosure can store the risk values ​​determined by each module in the task chain after completing its corresponding task, so that when performing rollback or recalculation operations, it can quickly determine which module's risk value to use to update the global risk value.

[0198] In box 311, the robot can terminate the execution of the task chain.

[0199] If the robot fails to roll back or exceeds the limit, the robot can terminate the execution of the task chain.

[0200] It should be noted that if the global risk value exceeds a predetermined threshold d at any stage of task execution, the task chain can be terminated directly. d can be set according to requirements. When the global risk value exceeds the threshold d, it indicates that the risk associated with the current task chain is completely unacceptable. To ensure safety, the robot can immediately terminate the task chain. The threshold d can be greater than threshold a, threshold b, etc.

[0201] In box 312, the robot determines whether it is the final task.

[0202] In box 313, the robot begins the next task.

[0203] As an example, if the robot operates via boxes 301-305, then the next task m+1 of task m is initiated.

[0204] As another example, if the robot operates via boxes 301-304, 306-308, and 305, then the next task m+1 of task m is started.

[0205] As another example, if the robot operates via boxes 301-304, 306-310, and 305, then the next task l+1 of task l is initiated.

[0206] In box 314, the robot confirms that the task chain has been executed successfully.

[0207] Based on this approach, the embodiments of this disclosure can perform real-time uncertainty assessment and risk prediction for the modules executing tasks. If the risk prediction result is unacceptable, a review and verification of this module is initiated. If the risk result is still unacceptable after the review and verification, the process reverts to the preceding module that may have resulted in an unacceptable risk result to re-execute the preceding task. This can effectively suppress the cascading diffusion of uncertainty in the task chain, reduce the execution risk of the task chain, and ensure the accuracy of the execution results corresponding to the task chain.

[0208] Example devices and equipment

[0209] Embodiments of this disclosure also provide corresponding apparatus for implementing the above methods or processes. Figure 4 A schematic structural block diagram of an apparatus 400 for executing a task chain according to certain embodiments of the present disclosure is shown. The apparatus 400 may be implemented as or included in the task processing device 110 discussed above. The various modules / components in the apparatus 400 may be implemented by hardware, software, firmware, or any combination thereof.

[0210] like Figure 4 As shown, the apparatus 400 includes a first determining module 410 configured to determine first evaluation information in response to a first task in a task chain being executed by a first module, the first evaluation information indicating the uncertainty of the first module executing the first task; a constructing module 420 configured to construct a plurality of perturbation samples corresponding to the input of the first task by processing the input of the first task in response to the first evaluation information satisfying a first condition; a providing module 430 configured to provide the plurality of perturbation samples to the first module to determine second evaluation information based on the plurality of processing results of the plurality of perturbation samples; a second determining module 440 configured to determine a second task from at least one prior task in the task chain that was executed before the first task in response to the second evaluation information satisfying a second condition; and a first triggering module 450 configured to trigger the second module to re-execute the second task.

[0211] In some embodiments, the apparatus 400 further includes a second triggering module configured to: trigger a third module to execute a third task in a task chain in response to the first evaluation information not meeting a first condition or in response to the second evaluation information not meeting a second condition; and an update module configured to update global evaluation information associated with the task chain based on the first evaluation information and / or the second evaluation information.

[0212] In some embodiments, the apparatus 400 further includes a result determination module configured to: in response to the second evaluation information not meeting the second condition, determine the output result of the first task based on multiple processing results.

[0213] In some embodiments, the apparatus 400 further includes a termination module configured to terminate the execution of the task chain in response to the global evaluation information satisfying a third condition; or a third trigger module configured to trigger a fourth module to re-execute the fourth task in the task chain in response to the global evaluation information satisfying a fourth condition, wherein the fourth task is executed before the second task.

[0214] In some embodiments, the second triggering module is further configured to: determine an initial risk value indicated by the global evaluation information; apply a predetermined forgetting coefficient to decay the initial risk value to determine a first risk value; and determine an updated risk value of the global evaluation information based on the first risk value and a second risk value determined based on the first evaluation information and / or the second evaluation information.

[0215] In some embodiments, the first determining module 410 is configured to, in response to the execution of the first task, determine an uncertainty measure corresponding to the first task; perform normalization processing on the uncertainty measure to determine a risk value corresponding to the first task; and determine first evaluation information based on the risk value.

[0216] In some embodiments, the first determining module 410 is configured to determine a normalization strategy that matches the task type from a plurality of preset normalization strategies based on the task type of the first task; and to perform normalization processing on the uncertainty measure based on the normalization strategy to determine the risk value corresponding to the first task.

[0217] In some embodiments, the construction module 420 is configured to perform at least one of the following: perform geometric transformation and / or scaling transformation on the image content in the input; add color perturbation to the image content in the input; and rewrite the prompt words in the input.

[0218] In some embodiments, module 430 is provided and configured to: acquire multiple processing results generated by the first module processing multiple perturbation samples; and determine second evaluation information based on the degree of uncertainty corresponding to the multiple processing results and / or the differences between the multiple processing results.

[0219] In some embodiments, the apparatus 400 further includes a third determining module configured to: determine that the second evaluation information satisfies the second condition in response to the uncertainty level reaching a first threshold; or determine that the second evaluation information satisfies the second condition in response to the difference reaching a second threshold.

[0220] In some embodiments, the apparatus 400 further includes a fourth determining module configured to: determine a plurality of output probability distributions corresponding to a plurality of processing results; a fifth determining module configured to: determine the average distribution of the plurality of output probability distributions; and a sixth determining module configured to: determine the degree of uncertainty corresponding to the plurality of processing results based on the normalized entropy of the average distribution.

[0221] In some embodiments, the second determining module 440 is configured to: determine a second task from at least one prior task based on evaluation information corresponding to at least one prior task; or determine the previous task corresponding to the first task among at least one prior task as the second task.

[0222] In some embodiments, the task chain includes multiple tasks corresponding to the robot pipeline, and the first module and the second module are modules based on machine learning models.

[0223] The units included in device 400 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units may be implemented using software and / or firmware, such as machine-executable instructions stored on a storage medium. In addition to or as an alternative to machine-executable instructions, device 400 or some or all of its units may be implemented at least partially by one or more hardware logic components. By way of example and not limitation, exemplary types of hardware logic components that may be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chips (SoCs), complex programmable logic devices (CPLDs), and so on.

[0224] Figure 5 A block diagram of an electronic device 500 in which one or more embodiments of the present disclosure may be implemented is shown. It should be understood that... Figure 5 The electronic device 500 shown is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. Figure 5 The electronic device 500 shown can be used to achieve Figure 1 The task processing device 110 shown.

[0225] like Figure 5As shown, electronic device 500 is in the form of a general-purpose electronic device. Components of electronic device 500 may include, but are not limited to, one or more processors 510 or processing units, memory 520, storage device 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. Processor 510 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 520. In a multiprocessor system, multiple processors execute computer-executable instructions in parallel to improve the parallel processing capability of electronic device 500.

[0226] Electronic device 500 typically includes multiple computer storage media. Such media can be any accessible media that is accessible to electronic device 500, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 520 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 530 can be a removable or non-removable medium and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data (e.g., training data for training) and can be accessed within electronic device 500.

[0227] Electronic device 500 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 5 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 520 may include computer program product 525 having one or more program modules configured to perform various methods or actions of various embodiments of this disclosure.

[0228] Communication unit 540 enables communication with other electronic devices via a communication medium. Additionally, the functionality of components of electronic device 500 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, electronic device 500 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0229] Input device 550 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 560 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 500 can also communicate with one or more external devices (not shown) via communication unit 540 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 500, or with any device that enables electronic device 500 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interface (not shown).

[0230] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0231] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0232] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0233] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0234] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0235] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method of executing a chain of tasks, characterized by, The method comprises: in response to a first task in a task chain being executed by a first module, determining first evaluation information indicating uncertainty of the first module executing the first task; in response to the first evaluation information satisfying a first condition, constructing a plurality of perturbation samples corresponding to an input of the first task by processing the input; providing the plurality of perturbation samples to the first module to determine second evaluation information based on a plurality of processing results of the plurality of perturbation samples; in response to the second evaluation information satisfying a second condition, determining a second task from at least one preceding task executed before the first task in the task chain; and triggering a second module to re-execute the second task; wherein the determining of the second task from the at least one preceding task executed before the first task in the task chain comprises: determining the second task from the at least one preceding task based on evaluation information corresponding to the at least one preceding task.

2. The method of claim 1, wherein, The method further comprises: in response to the first evaluation information not satisfying the first condition or in response to the second evaluation information not satisfying the second condition, triggering a third module to execute a third task in the task chain; and updating global evaluation information associated with the task chain based on the first evaluation information and / or the second evaluation information.

3. The method of claim 2, wherein, The method further comprises: in response to the second evaluation information not satisfying the second condition, determining an output result of the first task based on the plurality of processing results.

4. The method of claim 2, wherein, The method further comprises: in response to the global evaluation information satisfying a third condition, terminating execution of the task chain; or in response to the global evaluation information satisfying a fourth condition, triggering a fourth module to re-execute a fourth task in the task chain, the fourth task being executed before the second task.

5. The method of claim 2, wherein, The updating of the global evaluation information associated with the task chain based on the first evaluation information and / or the second evaluation information comprises: determining an initial risk value indicated by the global evaluation information; applying a predetermined forgetting coefficient to perform decay processing on the initial risk value to determine a first risk value; determining an updated risk value of the global evaluation information based on the first risk value and a second risk value determined based on the first evaluation information and / or the second evaluation information.

6. The method of claim 1, wherein, The determining of the first evaluation information comprises: in response to the first task being executed, determining an uncertainty measure corresponding to the first task; performing normalization processing on the uncertainty measure to determine a risk value corresponding to the first task; and determining the first evaluation information based on the risk value.

7. The method of claim 6, wherein, The performing of the normalization processing on the uncertainty measure to determine the risk value corresponding to the first task comprises: based on a task type of the first task, determining a normalization strategy matching the task type from a plurality of preset normalization strategies; and based on the normalization strategy, performing normalization processing on the uncertainty measure to determine the risk value corresponding to the first task. The method further comprises: in response to the first evaluation information not satisfying the first condition or in response to the second evaluation information not satisfying the second condition, triggering a third module to execute a third task in the task chain; and updating global evaluation information associated with the task chain based on the first evaluation information and / or the second evaluation information. The method further comprises: in response to the second evaluation information not satisfying the second condition, determining an output result of the first task based on the plurality of processing results. The method further comprises: in response to the global evaluation information satisfying a third condition, terminating execution of the task chain; or in response to the global evaluation information satisfying a fourth condition, triggering a fourth module to re-execute a fourth task in the task chain, the fourth task being executed before the second task. The updating of the global evaluation information associated with the task chain based on the first evaluation information and / or the second evaluation information comprises: determining an initial risk value indicated by the global evaluation information; applying a predetermined forgetting coefficient to perform decay processing on the initial risk value to determine a first risk value; determining an updated risk value of the global evaluation information based on the first risk value and a second risk value determined based on the first evaluation information and / or the second evaluation information. The determining of the first evaluation information comprises: in response to the first task being executed, determining an uncertainty measure corresponding to the first task; performing normalization processing on the uncertainty measure to determine a risk value corresponding to the first task; and determining the first evaluation information based on the risk value. The performing of the normalization processing on the uncertainty measure to determine the risk value corresponding to the first task comprises: based on a task type of the first task, determining a normalization strategy matching the task type from a plurality of preset normalization strategies; and based on the normalization strategy, performing normalization processing on the uncertainty measure to determine the risk value corresponding to the first task.

8. The method of claim 1, wherein, The constructing the plurality of perturbed samples corresponding to the input of the first task comprises at least one of: performing geometric transformation and / or scale transformation on image content in the input; adding color perturbation to the image content in the input; rewriting prompt words in the input.

9. The method of claim 1, wherein, The providing the plurality of perturbed samples to the first module to determine second evaluation information based on a plurality of processing results of the plurality of perturbed samples comprises: obtaining the plurality of processing results generated by the first module processing the plurality of perturbed samples; and determining the second evaluation information based on uncertainty degrees corresponding to the plurality of processing results and / or differences between the plurality of processing results.

10. The method of claim 9, wherein, The method further comprises: in response to the uncertainty degrees reaching a first threshold, determining that the second evaluation information satisfies the second condition; or in response to the differences reaching a second threshold, determining that the second evaluation information satisfies the second condition.

11. The method of claim 9, wherein, The method further comprises: determining a plurality of output probability distributions corresponding to the plurality of processing results; determining an average distribution of the plurality of output probability distributions; and determining the uncertainty degrees corresponding to the plurality of processing results based on a normalized entropy of the average distribution.

12. The method of claim 1, wherein, The task chain comprises a plurality of tasks corresponding to a robot pipeline, and the first module and the second module are modules based on machine learning models.

13. An apparatus for performing a chain of tasks, the apparatus comprising: The apparatus comprises: a first determining module configured to, in response to a first task in a task chain being executed by a first module, determine first evaluation information indicating uncertainty of the first module executing the first task; a constructing module configured to, in response to the first evaluation information satisfying a first condition, construct a plurality of perturbed samples corresponding to an input of the first task by processing the input; a providing module configured to provide the plurality of perturbed samples to the first module to determine second evaluation information based on a plurality of processing results of the plurality of perturbed samples; a second determining module configured to, in response to the second evaluation information satisfying a second condition, determine a second task from at least one preceding task preceding the first task in the task chain; and a first triggering module configured to trigger a second module to re-execute the second task; wherein the second determining module is further configured to determine the second task from the at least one preceding task based on evaluation information corresponding to the at least one preceding task.

14. An electronic device, comprising: The electronic device comprises: at least one processor; and at least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor, the instructions, when executed by the at least one processor, causing the electronic device to perform the method according to any one of claims 1 to 12.

15. A computer-readable storage medium having stored thereon computer- executable instructions, wherein, The computer-executable instructions, when executed by a processor, perform the method according to any one of claims 1 to 12.

16. A computer program product, comprising computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a processor, perform the method according to any one of claims 1 to 12.

Images