Task processing method and related system

By identifying the output of the initial task model and adapting the weight update strategy, the performance degradation and coupling of unknown task processing models in the existing technology is solved, and efficient and pluggable unknown task processing is achieved, reducing costs and improving the model Adaptability and reliability.

WO2024230139A1PCT designated stage expired Publication Date: 2024-11-14HUAWEI CLOUD COMPUTING TECHNOLOGIES CO LTD

Patent Information

Application Number
PCT/CN2023/136072
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-09-27
Filing Date
2023-12-04
Publication Date
2024-11-14

AI Technical Summary

Technical Problem

When the prior art deals with unknown tasks, the model performance drops sharply, resulting in unpredictable accidents and economic losses. Unknown task processing methods are usually highly coupled with specific task models, making it difficult to make the model pluggable and unsatisfied and unable to meet business needs.

Method used

By identifying the output of the initial task model and adapting the weight update strategy, any type of initial task model is implemented as input for unknown task training based on instance re-empowerment, decoupling the processing logic and code levels, and has the characteristics of pluggable models, and Improve training efficiency and model reliability through performance evaluation metrics and reliability evaluation.

Benefits of technology

It realizes efficient processing of unknown tasks, reduces labor and economic costs, improves the adaptability and versatility of the model, can quickly obtain a task model that meets business needs, and provides a model reliability feedback mechanism.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2023136072_14112024_PF_FP_ABST
    Figure CN2023136072_14112024_PF_FP_ABST
Patent Text Reader

Abstract

Provided in the present application is a task processing method, which is applied to a task processing system. The task processing system is used for providing a task model for processing an unknown task, and processing the unknown task on the basis of the task model. The method comprises: receiving an initial task model, a known task sample and an unknown task sample, which are input by a user; inputting at least one task sample into the initial task model for inference; determining a weight update strategy according to an inference result; inputting the known task sample and the unknown task sample into the initial task model; and performing model training by means of the weight update strategy, so as to obtain a task model. In the method, the weight update strategy is matched by means of the inference result of the initial task model for a task sample, such that unknown-task training based on instance reweighting can be performed by using any type of initial task model as an input, and the characteristic of a model being pluggable is provided.
Need to check novelty before this filing date? Find Prior Art

Description

A task processing method and related system

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on May 6, 2023, with application number 202310505723.6 and invention name “A task processing method and related system”, and claims priority to the Chinese patent application filed with the State Intellectual Property Office on September 27, 2023, with application number 202311264690.7 and invention name “A task processing method and related system”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of artificial intelligence (AI) technology, and in particular to a task processing method, system, computing device cluster, computer-readable storage medium, and computer program product. Background Art

[0003] In recent years, data-driven artificial intelligence (AI) technology has exploded with vitality across various fields. Supervised deep learning models (also referred to as task models or models) achieve good performance when processing samples within the training data distribution (also known as known tasks). However, in the real world, models may encounter data outside the training data distribution (also known as unknown tasks) at any time, causing a sharp decline in model performance and leading to unpredictable accidents and losses. For example, in industrial quality inspection, variations in raw materials, manufacturing processes, and the quality inspection environment can cause a drop in model detection performance by up to 80%. This can not only lead to missed delivery dates for industrial products and significant economic losses, but in severe cases, can even cause safety incidents on industrial production lines.

[0004] For unknown tasks, it's often necessary to collect and manually label large amounts of unknown sample data, remove the model from the business system, and finally retrain the model to adapt it to the unknown task. The collection and labeling of large amounts of unknown sample data incurs significant human and time costs, and the resulting stagnation of business systems can also lead to significant financial costs.

[0005] To address this, the industry has proposed several methods for handling unknown tasks. For example, these methods adjust the weights of known and unknown tasks during training to improve the task model's ability to handle unknown tasks. However, these methods are often highly coupled to the specific task model at the processing logic and code level. Once the task model changes or loses its structure, these methods become ineffective and fail to meet business needs.

[0006] Summary of the Invention

[0007] This application provides a task processing method that identifies the output of an initial task model (e.g., the initial task model's inference results on a task sample) and adapts a weight update strategy to enable instance-based reweighting of unknown tasks using any type of initial task model as input, thus achieving pluggable model performance. This application also provides a task processing system, computing device cluster, computer-readable storage medium, and computer program product corresponding to the above method.

[0008] In a first aspect, the present application provides a task processing method. The method is applied to a task processing system. The task processing system is used to provide a task model for processing unknown tasks, and to process unknown tasks based on the task model. The task processing system can be a software system, which can be deployed in a computing device cluster, for example, in a cloud computing cluster provided by a cloud service provider, or in an edge computing cluster. The computing device cluster executes the program code of the software system, thereby executing the task processing method of the present application. In some possible implementations, the task processing system can also be a hardware system, for example, a computing device cluster with unknown task processing capabilities. When the hardware system is running, the task processing method of the present application is executed.

[0009] Specifically, the task processing system can receive the initial task model, known task samples, and unknown task samples input by the user, input at least one task sample into the initial task model for inference, determine the weight update strategy based on the inference results, and then input the known task samples and unknown task samples into the initial task model, perform model training through the weight update strategy, and obtain the task model.

[0010] This method identifies the output of the initial task model (for example, the reasoning result of the initial task model on the task sample) and adapts the weight update strategy, so that any type of initial task model can be used as input to perform instance-based reweighting of unknown tasks. The unknown task processing method is decoupled from the task model (or model structure) at the processing logic and code levels. Even if the task model changes or the task model no longer has a certain structure, a task model that is effective for unknown task processing can be trained, and the model is pluggable.

[0011] In some possible implementations, known task samples are samples where the initial task model successfully infers, and unknown task samples are samples where the initial task model fails inference. The initial task model can be a business model taken offline from a business system and retrained to accommodate unknown tasks. This method retrains the initial task model using known task samples where the initial task model successfully infers, and unknown task samples where the initial task model fails inference. This method can quickly obtain a task model capable of effectively handling unknown tasks, meeting business requirements.

[0012] In some possible implementations, the task processing system may also determine a performance evaluation indicator based on the inference results. The task processing system may iteratively update the weights of the initial task model using a weight update strategy based on samples in the training set to obtain a transient model, and may perform a performance evaluation of the transient model using the performance evaluation indicator based on samples in the validation set to obtain a performance value, wherein the training set includes a first sample from the unknown task sample and a known task sample, and the validation set includes a second sample from the unknown task sample. When the performance value of the transient model reaches a performance threshold or the number of iterations of the initial task model reaches a maximum number, the task processing system outputs the transient model as the task model.

[0013] This method also adapts performance evaluation indicators based on the inference results. Appropriate performance evaluation indicators can accelerate the training efficiency of the task model and enable the task model that meets the requirements to be obtained more quickly.

[0014] In some possible implementations, the task processing system can also determine a model reliability evaluation metric based on the inference results. Then, based on samples in the test set, the task model's reliability can be determined using this model reliability evaluation metric. This allows for feedback to be provided to the user on the status of unknown task processing. If the reliability of the task model is low, the user can be encouraged to collect and annotate more unknown task samples in order to train a task model that is sufficiently effective for the current unknown task.

[0015] In some possible implementations, the task processing system can determine the task type based on the inference result and determine the weight update strategy corresponding to the task type. Task types include classification tasks, detection tasks, or segmentation tasks. The weight update strategy corresponding to classification tasks includes a sample-level update strategy, the weight update strategy corresponding to detection tasks includes an instance-level update strategy, and the weight update strategy corresponding to segmentation tasks includes a pixel-level update strategy.

[0016] This method identifies the task type based on the inference results and adapts the corresponding weight update strategy according to the task type. Using this weight update strategy to update the model weights can make the model converge faster, thereby accelerating training efficiency.

[0017] In some possible implementations, the performance evaluation indicators corresponding to the classification task include the F value, the model reliability evaluation indicators corresponding to the classification task include the sample classification error rate, the performance evaluation indicators corresponding to the detection task include the mean average precision mAP, the model reliability evaluation indicators corresponding to the detection task include the instance regression error rate and the instance classification error rate, the performance evaluation indicators corresponding to the segmentation task include the mean intersection-over-union (mIOU), and the model reliability evaluation indicators corresponding to the segmentation task include the pixel classification error rate.

[0018] This method can provide richer reference information for model training by adapting the corresponding performance evaluation indicators and model reliability evaluation indicators to the task type.

[0019] In some possible implementations, the task processing system can also obtain a preprocessing strategy based on the unknown task sample, the initial task model and the task type determined by the reasoning results, and then use the preprocessing strategy to enhance the unknown task to be processed, and process the enhanced unknown task through the task model.

[0020] This method can use any type of task model as input for inference-stage policy search, making it pluggable. Furthermore, by introducing preprocessing strategies for data augmentation during the task model inference phase, it can improve the task model's detection performance for unknown tasks in real time. Furthermore, both unknown task training and unknown task policy search achieve model-independent unknown task processing, improving the deployability and versatility of the entire unknown task processing system.

[0021] In some possible implementations, the task processing system may process original samples in a validation set using a sample processing strategy to obtain enhanced samples, where the original samples include a second sample from the unknown task sample. The task processing system may then obtain a preprocessing strategy based on the inference results of the initial task model on the original samples and the enhanced samples. The preprocessing strategy includes a sample processing strategy in which a second performance value of reasoning on the enhanced sample is greater than a first performance value of reasoning on the original sample, where the performance value is an indicator value of a performance evaluation indicator determined according to the task type.

[0022] This method introduces an external enhancement library (such as a library that provides sample processing strategies) and uses optimization methods to search for an effective set of data enhancement strategies and enhancement strengths for the task model, thereby improving the gain effect of data enhancement on the task model.

[0023] In some possible implementations, the task processing system may also fuse the reasoning results of the original sample and the enhanced sample according to the post-processing method corresponding to the task type, and the fused reasoning results are used to determine the second performance value. The fused reasoning results have higher reliability, and the performance values ​​determined based on the fused reasoning results are also more accurate, thereby improving the accuracy of the strategy search. It should be noted that this application also supports the integration of the results of multiple enhanced samples through voting and other methods, as a supplement to the introduced enhancement method, which can effectively filter out redundant results.

[0024] In some possible implementations, the task processing system may also process samples in the test set according to the sample processing strategy to obtain processed test samples, and determine the strategy reliability based on the inference results of the initial task model on the processed test samples and the labeling information of the samples in the test set.

[0025] This method uses an additional output, strategy reliability, to indicate the confidence level of the searched strategy in processing unknown tasks. This allows the user to input more sample processing strategies when the reliability is low, thereby searching for a preprocessing strategy that is sufficiently effective for the unknown task.

[0026] In some possible implementations, sample processing strategies are sampled from the search space, and the task processing system can also receive user-defined sample processing strategies and add them to the search space. This can expand the strategy search capability and provide greater flexibility.

[0027] In a second aspect, the present application provides a task processing system. The task processing system is used to provide a task model for processing unknown tasks, and processes the unknown tasks based on the task model. The system includes:

[0028] The interactive module is used to receive the initial task model, known task samples, and unknown task samples input by the user;

[0029] The unknown task training module is used to input at least one task sample into the initial task model for inference, determine the weight update strategy based on the inference result, input the known task sample and the unknown task sample into the initial task model, perform model training through the weight update strategy, and obtain the task model.

[0030] In some possible implementations, the known task samples are samples that are successfully inferred by the initial task model, and the unknown task samples are samples that fail to be inferred by the initial task model.

[0031] In some possible implementations, the unknown task training module is further configured to:

[0032] determining a performance evaluation indicator based on the reasoning result;

[0033] The unknown task training module is specifically used to:

[0034] Iteratively updating the weight of the initial task model using the weight update strategy according to samples in a training set to obtain a transient model, wherein the training set includes the first sample in the unknown task sample and the known task sample;

[0035] Performing a performance evaluation on the transient model using the performance evaluation indicator to obtain a performance value based on samples in a validation set, wherein the validation set includes a second sample in the unknown task sample;

[0036] When the performance value of the transient model reaches a performance threshold or the number of iterations of the initial task model reaches a maximum number, the transient model is output as the task model.

[0037] In some possible implementations, the unknown task training module is further configured to:

[0038] Determining a model reliability evaluation index based on the inference result;

[0039] The reliability of the task model is determined based on the samples in the test set using the model reliability evaluation index.

[0040] In some possible implementations, the unknown task training module is specifically configured to:

[0041] Determine the task type based on the inference result, and determine the weight update strategy corresponding to the task type;

[0042] The task type includes a classification task, a detection task or a segmentation task. The weight update strategy corresponding to the classification task includes a sample-level update strategy, the weight update strategy corresponding to the detection task includes an instance-level update strategy, and the weight update strategy corresponding to the segmentation task includes a pixel-level update strategy.

[0043] In some possible implementations, the performance evaluation indicators corresponding to the classification task include the F value, the model reliability evaluation indicators corresponding to the classification task include the sample classification error rate, the performance evaluation indicators corresponding to the detection task include the mean average precision mAP, the model reliability evaluation indicators corresponding to the detection task include the instance regression error rate and the instance classification error rate, the performance evaluation indicators corresponding to the segmentation task include the mean intersection-over-union (mIOU), and the model reliability evaluation indicators corresponding to the segmentation task include the pixel classification error rate.

[0044] In some possible implementations, the system further includes:

[0045] An unknown task strategy search module, configured to obtain a preprocessing strategy based on the unknown task sample, the initial task model, and the task type determined by the inference result;

[0046] An unknown task preprocessing module, configured to enhance the unknown task to be processed using the preprocessing strategy;

[0047] The unknown task processing module is used to process the enhanced unknown task through the task model.

[0048] In some possible implementations, the unknown task strategy search module is specifically configured to:

[0049] Processing the original samples in the validation set using a sample processing strategy to obtain enhanced samples, wherein the original samples include the second sample in the unknown task sample;

[0050] A preprocessing strategy is obtained based on the inference results of the initial task model on the original sample and the enhanced sample, wherein the preprocessing strategy includes a sample processing strategy in which a second performance value of the inference on the enhanced sample is greater than a first performance value of the inference on the original sample, and the performance value is an indicator value of a performance evaluation indicator determined according to the task type.

[0051] In some possible implementations, the unknown task strategy search module is further configured to:

[0052] According to the post-processing method corresponding to the task type, the inference results of the original sample and the enhanced sample are fused, and the fused inference results are used to determine the second performance value.

[0053] In some possible implementations, the unknown task strategy search module is further configured to:

[0054] Processing the samples in the test set according to the sample processing strategy to obtain processed test samples;

[0055] The strategy reliability is determined based on the inference results of the initial task model on the processed test samples and the labeling information of the samples in the test set.

[0056] In some possible implementations, the sample processing strategy is obtained by sampling from a search space, and the interaction module is further configured to:

[0057] receiving the user-defined sample processing strategy;

[0058] The unknown task strategy search module is further used to:

[0059] Add the user-defined sample processing strategy to the search space.

[0060] In a third aspect, the present application provides a computing device cluster. The computing device cluster includes at least one computing device, wherein the at least one computing device includes at least one processor and at least one memory. The at least one processor and the at least one memory communicate with each other. The at least one processor is configured to execute instructions stored in the at least one memory, so that the computing device or computing device cluster performs the task processing method described in the first aspect or any implementation of the first aspect.

[0061] In a fourth aspect, the present application provides a computer-readable storage medium storing instructions, wherein the instructions instruct a computing device or a computing device cluster to execute the task processing method described in the first aspect or any implementation of the first aspect.

[0062] In a fifth aspect, the present application provides a computer program product comprising instructions, which, when executed on a computing device or a computing device cluster, enables the computing device or computing device cluster to execute the task processing method described in the first aspect or any one of the implementations of the first aspect.

[0063] Based on the implementation methods provided in the above aspects, this application can also be further combined to provide more implementation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the technical methods of the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments.

[0065] FIG1 is a schematic diagram of the architecture of a task processing system provided in an embodiment of the present application;

[0066] FIG2 is a schematic diagram of a deployment form of a task processing system provided in an embodiment of the present application;

[0067] FIG3 is a flowchart of a task processing method provided in an embodiment of the present application;

[0068] FIG4 is a schematic diagram of a process of training a task model of an unknown task training module provided in an embodiment of the present application;

[0069] FIG5 is a schematic diagram of a flow chart of an unknown task strategy search module searching for a pre-processing strategy according to an embodiment of the present application;

[0070] FIG6 is a schematic diagram of a scenario in which a task processing method provided in an embodiment of the present application is applied to a task of visual inspection of surface defects of industrial products;

[0071] FIG7 is a schematic diagram of a scenario in which a task processing method improved in an embodiment of the present application is applied to an autonomous driving task or a visual inspection task for surface defects of industrial products;

[0072] FIG8 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0073] FIG9 is a schematic diagram of a computing device cluster provided in an embodiment of the present application;

[0074] FIG10 is a schematic diagram of another computing device cluster configuration provided by an embodiment of the application;

[0075] FIG11 is a schematic diagram of the structure of another computer cluster provided in an embodiment of the application. DETAILED DESCRIPTION

[0076] The terms "first" and "second" in the embodiments of this application are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Therefore, features specified as "first" or "second" may explicitly or implicitly include one or more of the features.

[0077] First, some technical terms involved in the embodiments of this application are introduced.

[0078] In the field of artificial intelligence (AI), a task model refers to a data-driven model used to process tasks to achieve a corresponding purpose or function. Specifically, a task model can be a deep learning model based on supervised learning. In some examples, the task model can be a classification model, object detection model, or segmentation model trained using supervised learning.

[0079] Unknown tasks, also known as unknown task data or unknown data, specifically refer to data outside the task model's knowledge distribution. Known tasks, also known as known task data or known data, specifically refer to data within the task model's knowledge distribution. The knowledge distribution can be the data distribution learned by the task model, such as the training data distribution.

[0080] While task models typically achieve good performance for known tasks, their performance degrades dramatically when handling unknown tasks. Using task models directly for unknown tasks can lead to unpredictable accidents and losses. For example, in industrial quality inspection, variations in raw materials, manufacturing processes, and the inspection environment can cause a task model's detection performance to drop by up to 80%. This can not only lead to missed product deliveries and financial losses, but can also cause safety incidents on industrial production lines.

[0081] Considering that collecting a large number of unknown task samples and labeling a large number of unknown task samples, and retraining the task model based on the labeled unknown task samples to adapt to the unknown tasks requires huge human resource costs and time costs, the industry has proposed some low-cost unknown task processing methods.

[0082] A typical approach to handling unknown tasks is instance-weighted unknown tasks. This method improves the task model's ability to handle unknown data by adjusting the weights of known and unknown tasks during training. Furthermore, this method can also alleviate the model's dependence on the amount of unknown task data, reducing the workload of collecting and labeling data.

[0083] Another typical approach to handling unknown tasks is based on inference-stage augmentation. This method applies different data augmentation transformations to unknown tasks during the inference phase, integrating inference to improve the task model's ability to handle unknown tasks. This approach eliminates the need for model retraining, thus avoiding the human and time costs associated with such retraining.

[0084] The aforementioned unknown task processing methods (those based on instance-based weighting and those based on inference-stage enhancement) are often highly coupled to specific task models at the processing logic and code levels, making them difficult to implement as pluggable models. Model pluggability refers to the ability to use any task model as input for an algorithm or technology, executing the algorithm or technology independently of the model to optimize the task model's weights or inference results.

[0085] Once the task model changes or the task model no longer has a certain structure, these non-model-independent technologies will become invalid or need to be readjusted. For example, in the process of multi-party collaborative development, once the business model is switched, or multiple parties use different business models, the above-mentioned unknown task processing methods will become invalid or need to be readjusted because the model cannot be pluggable. For example, in the multi-party collaborative development process of A, B, and C, C provides an unknown task processing solution for A and B, and A and B each have their own task models. If C's solution is coupled to a certain model, it will result in either C developing a set of unknown task processing methods for A and B's task models respectively, or A and B will need to adjust their own task models, making it difficult for multiple parties to work together.

[0086] In view of this, the present application provides a task processing method. The method can be executed by a task processing system. The task processing system is used to provide a task model for processing unknown tasks, and process unknown tasks based on the task model. The task processing system can be a software system, and the software system can be deployed in a computing device cluster, for example, in a cloud computing cluster provided by a cloud service provider, or in an edge computing cluster. The computing device cluster executes the program code of the software system, thereby executing the task processing method of the present application. In some possible implementations, the task processing system can also be a hardware system, for example, a computing device cluster with unknown task processing capabilities. When the hardware system is running, the task processing method of the present application is executed.

[0087] Specifically, the task processing system can receive the initial task model, known task samples, and unknown task samples input by the user, input at least one task sample into the initial task model for inference, determine the weight update strategy based on the inference results, and then input the known task samples and unknown task samples into the initial task model, perform model training through the weight update strategy, and obtain a task model for unknown task processing.

[0088] This method identifies the output of the initial task model (for example, the inference result of the initial task model on the task sample) and adapts the weight update strategy, thereby realizing unknown task training based on instance re-weighting with any type of initial task model as input, and has the feature of pluggable model. Furthermore, the method can also adapt the model reliability evaluation index based on the inference result of the initial task model on the task sample. Correspondingly, the reliability of the task model can also be output based on the model reliability evaluation index, thereby providing feedback to the user on the unknown task processing status. When the reliability of the task model is low, the user can be fed back to collect and label more unknown task samples in order to train a task model that is sufficiently effective for the current unknown task processing.

[0089] In addition, this method also supports the acquisition of preprocessing strategies. This preprocessing strategy can be based on the inference results of the task sample based on the initial task model, adapted to the performance evaluation indicators corresponding to the task type, and further adapted to the post-processing method corresponding to the task type, so that any type of task model can be used as input for the inference stage strategy search, with the feature of pluggable model. Similarly, this method can also adapt to the strategy reliability evaluation indicator. Accordingly, based on the strategy reliability evaluation indicator, the reliability of the preprocessing strategy can also be output, thereby providing feedback to the user. Furthermore, this method supports user-defined sample processing strategies. By adding user-defined sample processing strategies to the search space for strategy search, the ability to process unknown tasks can be expanded.

[0090] In order to make the technical solution of the present application clearer and easier to understand, the system architecture of the present application is introduced below with reference to the accompanying drawings.

[0091] Referring to the architectural diagram of the task processing system shown in FIG1 , the task processing system 10 includes an interaction module (not shown in FIG1 ) and an unknown task training module 102 . Furthermore, the task processing system 10 may also include an unknown task processing module 103 , an unknown task strategy search module 104 , and an unknown task pre-processing module 105 . Accordingly, the above modules are functional modules. The functional modules may be software modules. For example, when the task processing system 10 is an integrated software system, the interaction module is a module that provides interaction, typically a front-end module such as a user interface (UI) module. The unknown task training module 102 , the unknown task processing module 103 , the unknown task strategy search module 104 , and the unknown task pre-processing module 105 may be code blocks that implement corresponding business logic. For another example, when the task processing system 10 is a distributed software system, the interaction module, the unknown task training module 102 , and the unknown task strategy search module 104 may be microservices that implement corresponding functions. The functional modules may also be hardware modules, such as processors in a computing device or computing devices (such as servers) in a computing device cluster.

[0092] The inputs to the task processing system 10 may include an initial task model and training data, where the training data includes known task samples and unknown task samples. The task processing system 10 also includes unknown tasks to be processed, also known as unknown task inference data. In some possible implementations, the inputs to the task processing system 10 may also include user-defined sample processing strategies. The following describes each of the inputs to the task processing system 10:

[0093] Input 1: Initial task model. The initial task model is the task model to be trained. The trained task model is used to process the task. The initial task model can be a deep learning-based classification model, object detection model, segmentation model, or other machine learning model. In some implementations, the initial task model can be an offline business model from the business system, which is then retrained to adapt to unknown tasks.

[0094] Input 2: Training data. Training data can be a collection of known task samples and unknown task samples with labeled information. Known task samples are samples where the initial task model successfully infers (e.g., correctly inferred), while unknown task samples are samples where the initial task model fails inference (e.g., incorrectly inferred or unable to provide inference results).

[0095] Input 3: Unknown task reasoning data. Unknown task reasoning data can be abnormally distributed data encountered during the online operation of the task model (data outside the task model knowledge distribution), that is, unknown tasks to be processed.

[0096] Input 4: User-defined sample processing strategy. Users can customize one or more sample processing strategies, which can be organized in a list. Each element in the list can be a two-tuple, representing a user-defined sample processing function and the parameter value range of the function. It should be noted that the task processing method of the present application can also be executed without inputting a user-defined sample processing strategy. For example, a strategy search can be performed based on the system's built-in sample processing strategy.

[0097] The output of the task processing system 10 may include a task model and an unknown task inference result (e.g., the result of processing an unknown task using a trained task model). In some possible implementations, the output of the task processing system 10 may also include one or more of the reliability of the task model, the preprocessing strategy, and the reliability of the preprocessing strategy. The following describes each output of the task processing system 10:

[0098] Output 1: Task model. The task model includes model weights trained by the unknown task training module 102 and can adapt to the data distribution of the unknown task.

[0099] Output 2: Unknown task inference result. The unknown task inference result is usually related to the task (or task model). For example, if the input task model is a classification model, or the task is a classification task, the unknown task inference result can be the probability that the image belongs to each class.

[0100] Output 3: Reliability of the task model. The reliability of the task model can be characterized by the confidence of the model weight output by the unknown task training module 102 for unknown task processing. This output is related to the recognition result of the task model by the task processing system 10. For example, if the recognition result is a classification model, the reliability of the task model can be the classification error rate of the task model for unknown task samples in the test set. For another example, if the recognition result is a target detection model, the reliability of the task model can be the classification and regression error rate of the task model for each label box of the unknown task samples in the task test set. If the reliability is low, the user can provide and annotate more unknown task samples to the task processing system 10 to train a sufficiently effective task model.

[0101] Output 4: Preprocessing strategy. The preprocessing strategy can be a sample processing strategy that is helpful for unknown task reasoning, as found by the unknown task strategy search module 104. The sample processing strategy can include a sample enhancement method (which can be represented by a sample processing function) and strength (which can be represented by parameters of the sample processing function). The preprocessing strategy can be a list, where each element in the list is a sublist, and each element in the sublist is a two-tuple representing a specific sample processing function and strength.

[0102] Output 5: Reliability of the preprocessing strategy. The reliability of the preprocessing strategy can be represented by the confidence level of the preprocessing strategy output by the unknown task strategy search module 104. This output is related to the recognition result of the task model by the task processing system 10. For example, if the recognition result is a classification model, the reliability of the preprocessing strategy can be the classification error rate of the unknown task samples in the test set when the task model uses the preprocessing strategy to infer. When the reliability is low, the user can be fed back to provide a customized sample processing strategy to expand the search space in order to search for a preprocessing strategy that is sufficiently effective for the unknown task to be processed.

[0103] Next, the modules in the task processing system 10 , such as the interaction module, the unknown task training module 102 , the unknown task processing module 103 , the unknown task strategy search module 104 , and the unknown task preprocessing module 105 , are introduced.

[0104] The interaction module is used to receive the initial task model, known task samples, and unknown task samples input by the user. Specifically, the interaction module can provide an interaction interface to the user. The interaction interface can be a graphical user interface (GUI) or a command user interface (CUI). The user can input the initial task model, known task samples, and unknown task samples through an interaction interface such as a GUI or CUI. For example, the interaction interface can display a model list, and the user can select a model from the model list displayed on the interaction interface. The model selected by the user can be regarded as the initial task model input by the user. The user can input the path of the training data on the interaction interface, and the interaction module can obtain the training data based on the path input by the user. The training data can be a collection of known task samples and unknown task samples.

[0105] The unknown task training module 102 is used to input at least one task sample into the initial task model for inference, determine a weight update strategy based on the inference results, input known task samples and unknown task samples into the initial task model, and perform model training using the weight update strategy to obtain a task model. The unknown task processing module 103 is used to process the unknown task to be processed using the task model.

[0106] Furthermore, the unknown task strategy search module 104 can also be used to obtain a preprocessing strategy based on the unknown task sample, the initial task model, and the task type determined through inference results. The unknown task preprocessing module 105 is used to enhance the unknown task to be processed using this preprocessing strategy. Accordingly, the unknown task processing module 103 is specifically used to process the enhanced unknown task using the task model.

[0107] As shown in Figure 1, unknown task processing can be divided into a training phase and an inference phase. The following describes the training phase and the inference phase respectively.

[0108] During the training phase, the unknown task training module 102 performs model training based on the initial task model and training data (including known task samples and unknown task samples) input by the user to obtain a task model. Furthermore, the unknown task training module 102 can also output the reliability of the task model. The unknown task strategy search module 10 can search for a preprocessing strategy for the unknown task to be processed based on the initial task model and training data, such as the unknown task samples in the training data. Furthermore, the unknown task strategy search module 104 can also output the reliability of the preprocessing strategy.

[0109] During the inference phase, the unknown task processing module 103 processes the unknown task to be processed (unknown task inference data) using the task model output by the unknown task training module 102. Using the task model to process the unknown task to be processed can include using the task model to perform inference and outputting the unknown task inference result. In the example of FIG1 , the unknown task preprocessing module 105 can first enhance the unknown task to be processed using the preprocessing strategy output by the unknown task strategy search module 104. Accordingly, the unknown task processing module 103 processes the enhanced unknown task using the task model output by the unknown task training module 102.

[0110] Figure 1 introduces the architecture of the task processing system 10 of the embodiment of the present application. The following is an example of the deployment form of the task processing system 10. In some examples, the above-mentioned task processing system 10 can be deployed entirely in the cloud (cloud environment), or entirely on the edge (edge ​​environment), or on the terminal. In other examples, the task processing system 10 can also be deployed in different environments, for example, it can be deployed in the cloud and the edge in an edge-cloud collaborative manner. The following is an example of the deployment of the task processing system 10 in an edge-cloud collaborative manner.

[0111] Referring to FIG2 , which illustrates a schematic diagram of a deployment configuration for a task processing system 10, the deployment of the task processing system 10 may involve both cloud-based hardware devices and edge-side hardware devices. Cloud-based hardware devices typically have extensive computing power and significant storage capacity, such as servers and databases. Edge-side hardware devices have certain reasoning and storage capabilities, such as industrial computers and artificial intelligence edge computing devices. In some possible implementations, the deployment of the task processing system 10 also involves terminals, which may include IoT devices such as mobile phones, self-driving cars, and industrial quality inspection systems.

[0112] In the example of Figure 2, the unknown task training module 102 that supports pluggable models is deployed in the cloud. The input of the unknown task training module 102 can be the initial task model of the edge working node, the known task samples and unknown task samples stored in the cloud. The function of the unknown task training module 102 is to identify the input initial task model and adapt the corresponding weight update strategy according to the recognition result (such as the task type determined by the inference result of at least one task sample based on the initial task model). Furthermore, the corresponding performance evaluation indicators and model reliability evaluation indicators can also be adapted according to the recognition result. Then, the initial task model is trained based on the instance weighting method, and the weight of the trained task model is output, and the task model reliability is additionally output. The retrained task model can adapt to the unknown task data distribution and perform unknown task processing.

[0113] Similarly, the unknown task strategy search module 104 that supports pluggable models can also be deployed in the cloud. The input of the unknown task strategy search module 104 can be the initial task model of the edge working node and the unknown task sample stored in the cloud. The function of the unknown task strategy search module 104 is to identify the initial task model, call the adapted model evaluation index and post-processing method, search for sample processing strategies that are helpful for model reasoning, and thus output the preprocessing strategy. Furthermore, the unknown task strategy search module 104 can also additionally output the reliability of the preprocessing strategy. The preprocessing strategy searched by the unknown task strategy search module 104 can preprocess unknown samples when performing task reasoning at the edge, thereby improving the processing performance of the task model for unknown samples.

[0114] The unknown task processing module 103 and the unknown task preprocessing module 105 can be deployed at the edge. Among them, the input of the unknown task preprocessing module 105 can be the unknown task inference data (i.e., the unknown sample to be processed) provided by the terminal (such as the application running on the terminal) and the preprocessing strategy output by the cloud. The function of the unknown task preprocessing module 105 is to call the preprocessing strategy to preprocess the unknown task to be processed, and improve the processing performance of the subsequent task model for the unknown task. The input of the unknown task processing module 103 can be the unknown task sample enhanced by the unknown task preprocessing module 105 and the task model output by the cloud. The function of the unknown task processing module 103 is to call the task model to process the enhanced unknown task sample, for example, to call the task model to infer and fuse the enhanced unknown task sample, and return the unknown task inference result to the terminal (the application running on the terminal).

[0115] The interaction module (not shown in Figures 1 and 2) can be deployed on the cloud and edge. The interaction module deployed on the cloud is used to receive known task samples and unknown task samples, and can also receive the initial task model. The interaction module deployed on the edge is used to receive unknown tasks to be processed and can further display the inference results of unknown tasks.

[0116] It should be noted that FIG2 is only an example of the deployment form of the task processing system 10. In other possible implementations of the present application, the task processing system 10 may also adopt other deployment methods, and the present application does not limit this.

[0117] Based on the task processing system 10 shown in FIG1 , the present application further provides a task processing method. The task processing method of the present application is introduced below from the perspective of the task processing system 10 .

[0118] Referring to the flowchart of the task processing method shown in FIG3 , the method can be executed by the task processing system 10. The task processing system 10 is used to provide a task model for processing unknown tasks, and to process the unknown tasks based on the task model. The method includes:

[0119] S302 : The task processing system 10 receives an initial task model, known task samples, and unknown task samples input by a user.

[0120] The initial task model can be the user's own business model. The unknown task training module 102 in the task processing system 10 can adapt the user's own business model as needed. That is, the unknown task training module 102 is decoupled from the user's own business model. Therefore, the task processing system 10 can receive the initial task model input by the user and perform training based on the initial task model to obtain a task model for processing unknown tasks.

[0121] Known task samples can be in-distribution samples, and unknown task samples can be out-of-distribution samples. Known task samples and unknown task samples carry labeling information. Depending on the type of the initial task model or the task, the labeling information carried by the task samples (such as known task samples and unknown task samples) may be different. For example, when the initial task model is a classification model, the labeling information carried by the task samples may include category labels. For another example, when the initial task model is a target detection model, the labeling information carried by the task samples may include label boxes and category labels. The label box is used to indicate the location of the target.

[0122] S304 : The task processing system 10 inputs at least one task sample into the initial task model for inference, and determines a weight update strategy according to the inference result.

[0123] The task processing system 10 can extract at least one task sample (known or unknown) and perform inference using the task model input by the user to obtain an inference result of the initial task model on the task sample. The task processing system 10 can then identify the task based on the inference result, such as the format of the inference result, to obtain the task type. The task processing system 10 can then update the strategy based on the weight corresponding to the determined task type.

[0124] The task type is used to identify tasks of different categories. Specifically, the task type may include any of the following: a classification task, a detection task (such as a target detection task), or a segmentation task. When the inference result includes a category, or the inference result includes a category and a scalar with a value of 0-1 (characterizing the reliability of the category), the task type may be a classification task. When the inference result includes a category and a position coordinate, or the inference result includes a category, a position coordinate, and two scalars with a value of 0-1, the task type may be a detection task. When the inference result includes a matrix of the same size as the input image, and the elements in the matrix include categories, or include categories and a scalar with a value of 0-1, the task type may be a segmentation task.

[0125] The weight update strategy corresponding to the classification task includes a sample-level update strategy, the weight update strategy corresponding to the detection task includes an instance-level update strategy, and the weight update strategy corresponding to the segmentation task includes a pixel-level update strategy. Based on this, the task processing system 10 can determine the weight update strategy corresponding to the task type according to the mapping relationship between the task type and the weight update strategy. For example, when the task processing system 10 determines that the task type is a classification task, it can determine that the weight update strategy corresponding to the task type is a sample-level update strategy. Among them, the sample-level update strategy, the instance-level update strategy, and the pixel-level update strategy are update strategies of different granularities, and an instance can refer to a target detected in a sample.

[0126] S306 , the task processing system 10 inputs the known task samples and the unknown task samples into the initial task model, performs new model training through the weight update strategy, and obtains the task model.

[0127] Specifically, the task processing system 10 can input known task samples and unknown task samples into the initial task model, and iteratively update the weights based on the initial task model's reasoning results of the task samples and the sample's labeling information. When the model after weight update meets the set conditions, the model after weight update can be output as the task model to process unknown tasks. In particular, when updating the weights, the task processing system 10 can adopt a weight update strategy corresponding to the task type. For example, when the task type is a classification task, the task processing system 10 can adopt a sample-level weight update strategy to update the weights of the initial task model. In particular, the task processing system 10 adopts a sample-level weight update strategy to update the weights, specifically by calculating the loss value based on the reasoning results of the task samples, for example, calculating the loss value based on the reasoning results and labeling information of a batch of task samples (one or more task samples), and then updating the weights based on the loss value.

[0128] In some possible implementations, the performance evaluation indicators corresponding to the classification task include the F value (e.g., F1-score), the performance evaluation indicators corresponding to the detection task include the mean average precision (mAP), and the performance evaluation indicators corresponding to the segmentation task include the mean intersection over union (mIOU). mIOU is generally calculated based on the class. By taking the average of the IOU of each class, a global performance evaluation can be achieved. The task processing system 10 can also determine the performance evaluation indicator based on the inference results. For example, the task processing system 10 can determine the task type based on the inference results and then determine the performance evaluation indicator corresponding to the task type. Accordingly, the task processing system 10 can filter the task model from the trained transient model based on the performance evaluation indicator. For example, the task processing system 10 can divide the unknown task samples and the known task samples to obtain a training set and a validation set. The training set includes the first sample from the unknown task samples and the known task samples, and the validation set includes the second sample from the unknown task samples. The task processing system can iteratively update the weights of the initial task model using a weight update strategy based on the samples in the training set to obtain a transient model. The task processing system 10 can verify the transient model (e.g., a model that meets the training stop condition) on the validation set. Specifically, the task processing system 10 can evaluate the performance of the transient model using a performance evaluation indicator based on the samples in the validation set to obtain a performance value. When the performance value of the transient model reaches a performance threshold or the number of iterations of the initial task model reaches a maximum number of iterations, the transient model can be output as the task model. When the performance value does not reach the performance threshold or the number of iterations of the initial task model does not reach the maximum number of iterations, model training can continue.

[0129] In some possible implementations, the model reliability evaluation indicators corresponding to the classification task include the classification error rate (for example, the sample classification error), and the model reliability evaluation indicators corresponding to the detection task include the regression error rate (for example, the instance regression error rate) and the classification error rate (for example, the instance classification error rate). The model reliability evaluation indicators corresponding to the segmentation task include the classification error rate (for example, the pixel classification error rate). Accordingly, the task processing system 10 can also determine the model reliability evaluation indicators based on the reasoning results. For example, the task type can be determined based on the reasoning results, and then the model reliability evaluation indicators corresponding to the task type can be determined. The task processing system 10 can determine the reliability of the task model based on the samples in the test set through the model reliability evaluation indicators. Furthermore, the task processing system 10 can output the reliability of the task model.

[0130] S308 , the task processing system 10 obtains a preprocessing strategy according to the unknown task sample, the initial task model, and the task type determined by the inference result.

[0131] A preprocessing strategy is a strategy used to enhance unknown tasks during the inference phase. It can include at least one sample processing strategy, which consists of a sample processing function and its strength, expressed as follows: (function name N, strength S). ​​Function name N can also be a method name, and strength is the degree of deformation of the current sample processing strategy, ranging from 0 to 1. A sample processing strategy ('Flip', 0.5) indicates a 50% probability of horizontally flipping a sample, while a sample processing strategy ('Rotate', 0.3) rotates the sample clockwise by 108 degrees (0.6*360 degrees).

[0132] Specifically, the unknown task samples can be divided into a training set and a validation set. For example, the first sample in the unknown task samples is divided into the training set, and the second sample in the unknown task samples is divided into the validation set. The task processing system 10 can process the original samples in the validation set through the sample processing strategy to obtain enhanced samples. The original samples include the second sample in the unknown task samples, and the enhanced samples are samples processed using the sample processing strategy. The task processing system 10 can obtain a preprocessing strategy based on the inference results of the initial task model on the original samples and the enhanced samples. The preprocessing strategy includes a sample processing strategy in which the second performance value of the enhanced sample reasoning is greater than the first performance value of the original sample reasoning, wherein the performance value can be an index value of a performance evaluation index determined according to the task type.

[0133] In some possible implementations, the task processing system 10 may determine a search space, the search space including at least one sample processing strategy, and then sample a sample processing strategy from the search space, and use the sampled sample processing strategy to process the samples in the validation set to obtain an enhanced validation set. The task processing system 10 then obtains a first performance value (for example, for a detection task, the first performance value may be a first accuracy, or referred to as a first precision) based on the first inference result of the task model on the samples in the validation set, and obtains a second performance value based on the second inference result of the task model on the samples in the enhanced validation set. The task processing system 10 may determine a preprocessing strategy based on the first performance value and the second performance value. The preprocessing strategy may include a sample processing strategy in which the second performance value is greater than the first performance value.

[0134] Furthermore, the task processing system 10 may also determine a post-processing method corresponding to the task type. Post-processing methods may include, but are not limited to, maximum confidence and non-maximum suppression. Accordingly, the task processing system 10 may fuse the second inference result (the inference result for the enhanced sample) and the first inference result (the inference result for the original sample) according to the post-processing method corresponding to the task type, and then obtain a second performance value based on the fused inference result. In some possible implementations, the task processing system 10 may also determine the reliability of the pre-processing strategy and output the reliability of the pre-processing strategy. Specifically, the task processing system 10 may process samples in the test set according to the sample processing strategy to obtain processed test samples, and determine the strategy reliability based on the inference results of the initial task model on the processed test samples and the labeling information of the samples in the test set. For example, the task processing system 10 may process samples in the test set according to the sample processing strategy in which the second performance value is greater than the first performance value to obtain processed test samples, and then the task processing system 10 may infer the processed test samples according to the task model to obtain a third inference result, and determine the strategy reliability based on the third inference result and the labeling information of the samples in the test set.

[0135] Taking into account the situation where the system's built-in sample processing strategy is difficult to meet the needs, the task processing system 10 can also receive user-defined sample processing strategies. Then the task processing system 10 can add the user-defined sample processing strategies to the search space and perform strategy search, thereby flexibly expanding the unknown task processing capabilities.

[0136] It should be noted that S308 is merely an optional step in the embodiment of the present application. The unknown task processing method of the present application can be executed by obtaining a preprocessing strategy through other means. For example, the task processing system can obtain a preprocessing strategy based on the initial task model and unknown task samples without having to reuse the inference results or task types output during unknown task training. Alternatively, the unknown task processing method of the present application can be executed without obtaining a preprocessing strategy.

[0137] S310 , the task processing system 10 uses a pre-processing strategy to enhance the unknown task to be processed.

[0138] Specifically, the task processing system 10 may process the unknown task to be processed according to the sample processing function and strength in the preprocessing strategy, thereby enhancing the unknown task to be processed.

[0139] It should be noted that the task processing method of the embodiment of the present application may also not execute S308 to S310. For example, when processing unknown tasks to be processed, the task processing system 10 may input the unknown tasks to be processed into the task model for processing without performing pre-processing such as enhancement on the unknown tasks.

[0140] S312 , the task processing system 10 processes the enhanced unknown task through the task model.

[0141] Specifically, the task processing system 10 may input the enhanced unknown task into the task model, and the task model may infer the enhanced unknown task, thereby implementing unknown task processing. The task processing system 10 may output the unknown task inference result.

[0142] It should be noted that when the task processing system 10 does not execute S308 to S310, the task processing system 10 can directly input the unknown task to be processed into the task model, and the task model can infer the unknown task to be processed, thereby implementing unknown task processing. In addition, the task processing method of the present application can also be executed without executing S312. For example, if no unknown task is received within a period of time, the task processing system 10 can skip executing S312.

[0143] Based on the above description, it can be seen that the embodiment of the present application provides a model-independent (model pluggable) unknown task processing method, which adapts the weight update strategy by identifying the output of the initial task model (for example, the reasoning result of the task sample), thereby realizing the unknown task training based on instance re-empowerment with any type of initial task model as input, and has the feature of model pluggability. Furthermore, the method can also adapt the model reliability evaluation index, and accordingly, the reliability of the task model can also be output based on the model reliability evaluation index, thereby providing feedback to the user on the unknown task processing status. When the reliability of the task model is low, the user can be fed back to collect and label more unknown task samples to train a task model that is effective enough for the current unknown task processing. In addition, the embodiment of the present application also introduces an enhancement method in the task model reasoning stage to improve the detection performance of the task model for unknown tasks in real time.

[0144] Next, the model training and strategy search processes are described in detail from the perspectives of the unknown task training module 102 and the unknown task strategy search module 104 in the task processing system 10 .

[0145] First, referring to FIG4 , a flow chart of an unknown task training module 102 training a task model includes the following steps:

[0146] (1) The unknown task training module 102 obtains the known task samples (known data) and unknown task samples (unknown data) input by the user, divides the unknown task samples into a training set, a validation set, and a test set, and merges the training set including the unknown task samples with the known task samples into a new training set. Then, by calculating the proportion of the unknown task samples in the training set, the unknown task samples are assigned initialization weights.

[0147] To facilitate understanding, the weight initialization process is explained below with examples.

[0148] For a classification task, assuming that in the training data received by the unknown task training module 102, 1,000 known task samples and 20 unknown task samples are divided into the training set, and the ratio of unknown task samples to known task samples in the training set is 50:1, accordingly, the unknown task training module 102 can assign 50 times the weight to the unknown task samples.

[0149] For a target detection task, assuming that in the training data received by the unknown task training module 102, 1,000 known label frames and 50 unknown label frames are divided into a training set, and the ratio of known label frames to unknown label frames in the training set is 20:1, accordingly, the unknown task training module 102 can assign a 20-fold weight to the unknown task samples.

[0150] (2) The unknown task training module 102 extracts a single piece of reasoning data (e.g., a single task sample) from the training set, and uses the initial task model input by the user for reasoning, performs task identification on the output result of the initial task model (e.g., the reasoning result), and obtains the task type.

[0151] (3) The unknown task training module 102 automatically adapts the weight update strategy, performance evaluation index and model reliability evaluation index corresponding to the task type according to the recognition result (recognized task type) of step (2).

[0152] (4) The unknown task training module 102 uses the training set and weights generated in step (1) to train the initial task model and temporarily stores the trained task model weights.

[0153] The trained task model weights may be weights of the initial task model updated using a weight update strategy based on samples in the training set. Specifically, the unknown task training module 102 may input samples from the training set into the initial task model and update the weights of the initial task model using a weight update strategy. When the updated model meets the training stop condition, iterative training may be stopped to obtain the trained task model weights. The model after weight update is also referred to as a transient model.

[0154] (5) The unknown task training module 102 performs a performance evaluation on the trained model (transient model) based on the validation set generated in step (1) using the performance evaluation indicator adapted in step (3). When the performance of the trained model reaches the performance threshold or the number of iterations of the initial task model reaches the maximum number of iterations, the process proceeds to step (7); otherwise, the process proceeds to step (6) and continues iterative training.

[0155] (6) The unknown task training module 102 determines the new round of weights of the model on the training set based on the weight update strategy adapted in step (3), trains the model, temporarily stores the trained task model weights, and returns to step (5).

[0156] (7) The unknown task training module 102 uses the model reliability evaluation index adapted in step (3) to determine the reliability of the task model on the test set divided in step (1), and outputs the task model weight and the reliability of the task model.

[0157] FIG3 also shows the difference between the traditional unknown task training method and the unknown task training method of this application, as shown below:

[0158] The basic idea behind traditional unknown task training methods is to use weak classifiers to classify samples and continuously improve the classifier's accuracy through iterative weighting. In each iteration, the classifier's results and error rate are used as input, and the weight of each classifier is calculated. The sample weight is then updated based on the weight. Finally, the results of all classifiers are weighted and averaged to obtain the final classification result.

[0159] Compared with the traditional unknown task training method, the unknown task training module 102 of the embodiment of the present application can realize unknown task training based on instance re-weighting with any task model as input by identifying the task model output, adapting the corresponding weight update strategy, performance evaluation index and model reliability evaluation index, and has the feature of pluggable model. In addition, the unknown task training module 102 additionally outputs the reliability of the task model and provides feedback to the user on the status of unknown task processing. When the reliability of the task model is low, the user is fed back that they should collect and label more unknown task samples in order to train a task model that is effective enough for the current unknown task processing. In this way, the module solves the problem that the traditional unknown task training method cannot achieve pluggable model and unknown task processing status feedback, and has high business value.

[0160] Next, referring to FIG5 , a flow chart of an unknown task strategy search module 104 is shown, which includes the following steps:

[0161] (1) The unknown task strategy search module 104 obtains unknown task samples input by the user and divides the unknown task samples into a verification set and a test set.

[0162] It should be noted that the unknown task strategy search module 104 may also reuse the verification set and the test set in the embodiment shown in FIG. 4 .

[0163] (2) The unknown task strategy search module 104 extracts a single piece of reasoning data (for example, a single task sample) from the verification set, uses the initial task model input by the user for reasoning, performs task identification on the output result of the initial task model, and obtains the task type.

[0164] Similarly, the unknown task strategy search module 104 may also reuse the recognition result (eg, the recognized task type) of the initial task model in the embodiment shown in FIG. 4 .

[0165] (3) The unknown task strategy search module 104 adapts the performance evaluation index, post-processing method, and strategy reliability evaluation index corresponding to the task type according to the recognition result of step (2).

[0166] The performance evaluation index corresponding to the task type may also reuse the performance evaluation index determined by the unknown task training module 102 in the embodiment shown in FIG. 4 .

[0167] (4) When the user inputs a customized sample processing strategy, the unknown task strategy search module 104 merges the sample processing strategy input by the user with the default sample processing strategy to form a search space.

[0168] (5) The unknown task strategy search module 104 calculates the basic accuracy (eg, the first accuracy) of the input initial task model based on the validation set provided in step (1) and the performance evaluation index adapted in step (3).

[0169] (6) The unknown task strategy search module 104 samples a sample processing strategy from the search space, then uses the sampled sample processing strategy to perform data augmentation on the validation set, and then calls the input initial task model to perform inference on the augmented validation set and the original validation set, obtaining two sets of inference results. The inference results of the augmented validation set and the original validation set are fused based on the post-processing method adapted in step (3), and the accuracy of the fused result (e.g., the second accuracy) is calculated.

[0170] The sample processing strategy may include a sample processing function and strength. The unknown task strategy search module 104 may sample a sample processing strategy from the search space by random sampling or sequential sampling.

[0171] (7) If the new accuracy is less than or equal to the basic accuracy, the sampled sample processing strategy (sample processing function or strength) is considered unsuitable for the unknown task, and the unknown task strategy search module 104 resamples the strategy; if the new accuracy is greater than the basic accuracy, the unknown task strategy search module 104 subtracts the new accuracy from the basic accuracy to obtain the contribution of the sampled sample processing strategy, and stores the sampled sample processing strategy (sample processing function and strength) and contribution.

[0172] (8) The unknown task strategy search module 104 determines whether the maximum number of iterations has been reached. If not, return to step (6). If so, the unknown task strategy search module 104 first sorts the stored sample processing strategies by contribution, and retains strategies whose contribution satisfies the conditions (for example, it is greater than the preset contribution, or is ranked as top n, where n is a positive integer). The unknown task strategy search module 104 uses the retained sample processing strategies to enhance the test set in step (1), calls the input initial task model to reason on the enhanced test set and the original test set, fuses the reasoning results based on the post-processing method adapted in step (3), calculates the reliability of the preprocessing strategy, and outputs the preprocessing strategy of the unknown task and the reliability of the preprocessing strategy.

[0173] The preprocessing strategy for the unknown task may include a sample processing strategy whose contribution satisfies the above conditions.

[0174] (9) If the reliability of the pre-processing strategy is low, the unknown task strategy search module 104 can expand the unknown task processing capability based on the sample processing strategy updated by the user.

[0175] FIG5 also illustrates the difference between the traditional reasoning stage enhancement method and the reasoning stage enhancement method in the unknown task strategy search module 104 of the present application, as shown below:

[0176] The traditional enhancement method in the inference stage is to perform data enhancement operations such as horizontal flipping, contrast enhancement, and rotation angle on the original image during the model inference stage, obtain multiple images for inference separately, and then fuse and post-process the multiple results to obtain the final output result.

[0177] Compared with the traditional reasoning stage enhancement method, the unknown task strategy search module 104 proposed in the embodiment of the present application can realize the reasoning stage enhanced strategy search with any task model as input by identifying the task model output, adapting the corresponding post-processing method, performance evaluation index, and strategy reliability evaluation index, and has the feature of pluggable model. In addition, the unknown task strategy search module 104 has an additional optional input user's sample processing strategy and the reliability of the additional output pre-processing strategy. The user's sample processing strategy input increases the flexibility of the unknown task strategy search module 104. The user can use the built-in sample processing strategy of the unknown task strategy search module 104 to search for unknown task pre-processing strategies, or input a customized sample processing strategy to search together with the built-in sample processing strategy (such as the default sample processing strategy) to expand the unknown task processing capability.

[0178] The unknown task strategy search module 104 additionally outputs the reliability of the preprocessing strategy, which indicates the confidence level of the searched preprocessing strategy for processing unknown tasks. If the reliability is low, the module provides feedback to the user, requesting more sample processing strategies to search for a sufficiently effective unknown task preprocessing strategy for the unknown task. This module addresses the limitations of traditional inference-stage enhancement methods in achieving model pluggability, unknown task processing status feedback, and expanding unknown task processing capabilities, thus providing high business value.

[0179] It should be noted that the method steps of the embodiment shown in Figure 4 and the method steps of the embodiment shown in Figure 5 can be executed in parallel, or can be executed in sequence according to a set order. During the execution process, part of the data or intermediate results of the embodiment shown in Figure 4 or the embodiment shown in Figure 5 can be reused.

[0180] Next, the task processing method of this application is introduced in combination with specific application scenarios.

[0181] 6 shows a scenario diagram of a task processing method applied to a task of visual inspection of surface defects of industrial products. The task of visual inspection of surface defects of industrial products may include the following stages:

[0182] Phase 1: Data Collection: In this embodiment, an industrial camera collects red, green, and blue (RGB) images into a production database.

[0183] Phase 2: Data labeling: Defects are annotated on the RGB images of industrial product surfaces in the production database by a manual or automated labeling system, and the annotated images are stored in the training database.

[0184] Phase 3: Task model training and deployment. Use the labeled data (e.g., labeled images) in the training database to train the task model for quality inspection and deploy it into the quality inspection process.

[0185] Phase 4: Unknown tasks caused by changes in raw materials, production processes, quality inspection environment and other factors lead to a decline in the detection performance of the task model, and enter the unknown task processing process.

[0186] The unknown task processing process may include an unknown task training process and an unknown task preprocessing strategy search process.

[0187] The unknown task preprocessing strategy search process includes the following steps:

[0188] (1) The unknown task strategy search module 104 receives the initial task model for industrial quality inspection and unknown task samples input by the user. The unknown task samples can be divided into a validation set and a test set.

[0189] (2) The unknown task strategy search module 104 performs task identification. Specifically, the unknown task strategy search module 104 can call the model inference interface of the initial task model to infer the unknown task sample and determine the task type based on the output result. In some examples, if the output is a one-dimensional vector or scalar, the task type is identified as a classification task; if the output is a two-dimensional array or a nested list, the task type is identified as a target detection task; if the output is a higher-dimensional array, the task type is identified as a segmentation task. In this embodiment, the unknown task strategy search module 104 identifies the current task type as a target detection task.

[0190] (3) The unknown task strategy search module 104 performs task adaptation. Specifically, the unknown task strategy search module 104 adapts the performance evaluation index, post-processing method, and strategy reliability evaluation index based on the recognition result. For example, if the recognition result is a classification task, the unknown task strategy search module 104 determines that the performance evaluation index is F1-score, the post-processing method is maximum confidence, and the strategy reliability evaluation index is the classification error rate. In this embodiment, the recognition result is a target detection task, and accordingly, the performance evaluation index is Mean Average Precision, the post-processing method is non-maximum suppression, and the strategy reliability evaluation indicators are regression error rate and classification error rate.

[0191] (4) The unknown task strategy search module 104 determines the search space. If the user enters a customized sample processing strategy, it is merged into the default sample processing strategy space to form a search space. In this embodiment, the default sample processing strategy space includes 19 methods, namely: 'Color', 'Contrast', 'Brightness', 'Sharpness', 'AutoContrast', 'Equalize', 'Resize', 'Flip', 'Grey', 'Invert', 'Posterize', 'Solarize', 'HSV', 'Rotate', 'ShearX', 'ShearY', 'PepperNoise', 'GaussNoise', and 'Cutout'.

[0192] (5) The unknown task strategy search module 104 initializes the enhanced strategy list L, and uses the initial task model to infer the validation set based on the adapted Mean Average Precision evaluation indicator to obtain the basic accuracy P0.

[0193] (6) The unknown task strategy search module 104 randomly samples a sample processing strategy (also called data augmentation strategy M) from the search space in the format of (method name N, strength S). ​​The strength S is the degree of deformation of the current data augmentation strategy, ranging from 0 to 1. For example, ('Flip', 0.5) means that there is a 50% probability that a sample will be horizontally flipped; ('Rotate', 0.3) means that it will be rotated 108 degrees clockwise.

[0194] (7) The unknown task strategy search module 104 uses the initial task model to perform inference on the M-enhanced validation set and the original validation set, respectively. The inference results are fused based on the adapted non-maximum suppression post-processing method. Finally, the accuracy P1 of the fused result is evaluated based on the adapted Mean Average Precision indicator. If P1 is greater than P0, the contribution C1 = P1-P0 is recorded, and the data enhancement strategy M is updated to (method name N, strength S, contribution C) and stored in the enhancement strategy list L. If P1 is less than P0, it is determined whether the termination condition of the maximum number of iterations has been reached. If not, it returns to step (6) to continue iterating.

[0195] (8) The unknown task strategy search module 104 sorts the elements in the enhancement strategy list L in descending order according to the contribution C, retains the first K entries in the enhancement strategy list, and the initial task model uses the retained K strategies to calculate the strategy reliability on the test set. In this embodiment, the strategy reliability evaluation index is the regression error rate + classification error rate. The calculation process is as follows: 1) All images in the test set are looped. For a certain image I in the test set, the unknown task strategy search module 104 calls the strategy in the enhancement strategy list L to enhance I, generating Len(L) pieces of enhanced data, where Len(L) is the list length. 2) The initial task model infers Len(L) pieces of enhanced data and the original image I, and uses the non-maximum suppression post-processing method to fuse the inference results. 3) Calculate the error rate of the label box in image I. Taking the label box B belonging to category 1 as an example, obtain the prediction result R with the largest intersection-over-union ratio with B in the inference result. Assuming that the score of R belonging to category 1 is RS, and the intersection-over-union ratio of R and B is RI, then the classification reliability of the label box B is ConfidenceC=RS, and the regression reliability is ConfidencR=RI. 4) Determine the average classification reliability and the average regression reliability of all label boxes.

[0196] (9) Output the search-completed enhancement strategy list L (the preprocessing strategy includes the sample processing strategy in the enhancement strategy list L), as well as the classification reliability and regression reliability of the output preprocessing strategy.

[0197] The unknown task training process includes the following steps:

[0198] (1) The unknown task training module 102 receives the initial task model, known task samples and unknown task samples for industrial quality inspection input by the user, divides the unknown task samples into a training set, a validation set and a test set, and merges the training set including the unknown task samples with the known task samples into a new training set.

[0199] (2) The unknown task training module 102 performs task identification and task adaptation. In this embodiment, the unknown task training module 102 can inherit the recognition result of the unknown task strategy search module 104, and there is no need to perform repeated recognition. The unknown task training module 102 adapts the weight update strategy, performance evaluation index, and model reliability evaluation index related to the task. For example, if the recognition result is a classification task, the weight update strategy can be a sample-level weight update strategy, the performance evaluation index is F1-score, and the model reliability evaluation index is the classification error rate. In this embodiment, the recognition result is a target detection task, the weight update strategy is an instance-level weight update strategy, the performance evaluation index is mean Average Precision, and the model reliability evaluation index is the regression error rate and the classification error rate. It should be noted that the unknown task training module 102 can inherit the adaptation result of the unknown task strategy search module 104, for example, inherit the performance evaluation index adapted by the unknown task strategy search module 104.

[0200] (3) The unknown task training module 102 calculates the proportion of unknown task samples in the training set, assigns initialization weights to the unknown task samples, and temporarily stores the data weights.

[0201] (4) The unknown task training module 102 uses the temporarily stored data weights to train the input initial task model and temporarily stores the updated model weights.

[0202] (5) The unknown task training module 102 uses the adapted mAP indicator to evaluate the performance R of the updated model weights on the unknown task validation set.

[0203] (6) The unknown task training module 102 determines whether the maximum number of iterations has been reached or whether R is greater than the performance threshold. If not, an adapted weight update strategy is used to calculate the data weight for the next training of the task model. In this embodiment, the weight update strategy is an instance-level weight update strategy. Taking the label box B belonging to category 1 as an example, the prediction result P with the largest intersection-over-union ratio with B in the task model prediction result is obtained. Assuming that the score of P belonging to category 1 is PS, and the intersection-over-union ratio of P and B is PI, then the error rate of label box B ErrorB = ((1-PS) + (1-PI)) / 2. If B is the label of unknown data, then the weight of label box B in the next training is WeightB = 1 + ErrorB; if B is the label of known data, then the weight of label box B in the next training is WeightB = 1 - ErrorB. Calculate all training data label boxes in a loop to generate the data weight for the next task model training, temporarily store the data weight, and return to step (4).

[0204] (7) The unknown task training module 102 uses the adapted model reliability evaluation index to evaluate the reliability of the current model on the test set. This embodiment is a target detection task, and the strategy reliability evaluation index is regression reliability (characterized by regression error rate) and classification reliability (characterized by classification error rate). The calculation process is as follows: loop through all images in the test set, and for a certain image I in the test set, take the label box B belonging to category 1 as an example, obtain the prediction result R with the largest intersection-over-union ratio with B in the inference result, assuming that the score of R belonging to category 1 is RS, and the intersection-over-union ratio of R and B is RI, then the classification reliability of the label box B is ConfidenceC=RS, and the regression reliability is ConfidenceR=RI. Average the classification reliability and regression reliability of all label boxes.

[0205] (8) The unknown task training module 102 outputs the weight of the trained task model, as well as the classification reliability and regression reliability of the task model.

[0206] Refer to the scenario diagram of another task processing method applied to autonomous driving tasks or industrial product surface defect visual inspection tasks shown in Figure 7. In this scenario, the behavior of the internal modules of the task processing system 10 is similar to the embodiment shown in Figure 6, so the entire unknown task processing platform is represented here.

[0207] Different from the pure cloud architecture of the embodiment shown in FIG6 , the embodiment shown in FIG7 adopts an edge-cloud collaborative architecture. Specifically, the unknown task processing platform is located in the cloud, while the user is located at the edge. In this embodiment, the pluggable task processing system 10 receives the business data uploaded by the user with the help of the shared storage service in the cloud, and at the same time annotates the business data uploaded by the user in the cloud based on the annotation system, and prepares for unknown task training by training the initial task model uploaded by the user. The user does not need to know the internal details of the unknown task processing platform, but only needs to upload the initial task model to the unknown task processing platform, and the unknown task processing platform can return a task model that can adapt to the unknown task and a preprocessing strategy that is helpful for unknown task reasoning.

[0208] In addition, the processing object of the embodiment shown in FIG7 may be different from the processing object of the embodiment shown in FIG6. In this embodiment, the pluggable unknown task processing platform provides services in the cloud, and the number of users is unlimited. For example, it can be a multi-party collaboration scenario, while the embodiment shown in FIG6 is usually a unilateral development scenario. As shown in FIG7, the pluggable unknown task processing platform allows users to input the initial task model, so whether the user inputs the U-Net model based on autonomous driving segmentation or the YOLOX model based on industrial quality inspection, it can be processed after uploading to the pluggable unknown task processing platform, which solves the problem that the unknown task training technology based on model coupling and the inference stage enhancement technology are difficult to achieve model pluggability.

[0209] Based on the aforementioned task processing method, the present application further provides a task processing system 10. The task processing system 10 is used to provide a task model for processing unknown tasks, and process unknown tasks based on the task model. As shown in FIG1 , the task processing system 10 includes:

[0210] The interaction module (not shown in FIG1 ) is used to receive the initial task model, known task samples, and unknown task samples input by the user;

[0211] The unknown task training module 102 is used to input at least one task sample into the initial task model for inference, determine the weight update strategy based on the inference result, input the known task sample and the unknown task sample into the initial task model, perform model training through the weight update strategy, and obtain the task model.

[0212] For example, the above-mentioned interaction module and unknown task training module 102 can be implemented by hardware or software.

[0213] When implemented by software, the interaction module and the unknown task training module 102 can be applications running on a computing device (such as a server). For example, the unknown task training module 102 can be a computing engine running on a computing device, etc. The application can be provided to users in the form of a virtualization service. Virtualization services may include virtual machine (VM) services, bare metal server (BMS) services, and container services. Among them, the VM service can be a service that uses virtualization technology to virtualize a virtual machine (VM) resource pool on multiple physical hosts (such as computing devices) to provide users with VMs on demand for use. The BMS service is a service that virtualizes a BMS resource pool on multiple physical hosts to provide users with BMS on demand for use. The container service is a service that virtualizes a container resource pool on multiple physical hosts to provide users with containers on demand for use. VM is a simulated virtual computer, that is, a logical computer. BMS is a high-performance computing service that can be elastically scalable. Its computing performance is no different from that of a traditional physical machine and has the characteristics of secure physical isolation. Containers are a kernel virtualization technology that provides lightweight virtualization to isolate user spaces, processes, and resources. It should be understood that the VM service, BMS service, and container service mentioned above are merely examples. In actual applications, virtualization services can also be other lightweight or heavyweight virtualization services, which are not specifically limited here.

[0214] When implemented in hardware, the interaction module can be implemented by a transceiver or other transceiver module. The unknown task training module 102 can include at least one computing device, such as a server. Alternatively, the unknown task training module 102 can also be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD can be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0215] In some possible implementations, the known task samples are samples that are successfully inferred by the initial task model, and the unknown task samples are samples that fail to be inferred by the initial task model.

[0216] In some possible implementations, the unknown task training module 102 is further configured to:

[0217] determining a performance evaluation indicator based on the reasoning result;

[0218] The unknown task training module 102 is specifically used to:

[0219] Iteratively updating the weight of the initial task model using the weight update strategy according to samples in a training set to obtain a transient model, wherein the training set includes the first sample in the unknown task sample and the known task sample;

[0220] Performing a performance evaluation on the transient model using the performance evaluation indicator to obtain a performance value based on samples in a validation set, wherein the validation set includes a second sample in the unknown task sample;

[0221] When the performance value of the transient model reaches a performance threshold or the number of iterations of the initial task model reaches a maximum number, the transient model is output as the task model.

[0222] In some possible implementations, the unknown task training module 102 is further configured to:

[0223] Determining a model reliability evaluation index based on the inference result;

[0224] The reliability of the task model is determined based on the samples in the test set using the model reliability evaluation index.

[0225] In some possible implementations, the unknown task training module 102 is specifically configured to:

[0226] Determine the task type based on the inference result, and determine the weight update strategy corresponding to the task type;

[0227] The task type includes a classification task, a detection task or a segmentation task. The weight update strategy corresponding to the classification task includes a sample-level update strategy, the weight update strategy corresponding to the detection task includes an instance-level update strategy, and the weight update strategy corresponding to the segmentation task includes a pixel-level update strategy.

[0228] In some possible implementations, the performance evaluation indicators corresponding to the classification task include the F value, the model reliability evaluation indicators corresponding to the classification task include the sample classification error rate, the performance evaluation indicators corresponding to the detection task include the mean average precision mAP, the model reliability evaluation indicators corresponding to the detection task include the instance regression error rate and the instance classification error rate, the performance evaluation indicators corresponding to the segmentation task include the mean intersection-over-union (mIOU), and the model reliability evaluation indicators corresponding to the segmentation task include the pixel classification error rate.

[0229] In some possible implementations, the system 10 further includes:

[0230] An unknown task strategy search module 104 is configured to obtain a preprocessing strategy based on the unknown task sample, the initial task model, and the task type determined by the inference result;

[0231] An unknown task preprocessing module 105 is used to enhance the unknown task to be processed using the preprocessing strategy;

[0232] The unknown task processing module 103 is configured to process the enhanced unknown task using the task model.

[0233] Similar to the unknown task training module 102 , the unknown task processing module 103 , the unknown task strategy search module 104 , and the unknown task preprocessing module 105 may be implemented by hardware or software.

[0234] When implemented via software, the unknown task processing module 103, the unknown task policy search module 104, and the unknown task pre-processing module 105 can be applications running on a computing device (e.g., a server). For example, the unknown task processing module 103, the unknown task policy search module 104, and the unknown task pre-processing module 105 can be a computing engine running on the computing device. This application can be provided to users as a virtualized service.

[0235] When implemented through hardware, the unknown task processing module 103, the unknown task strategy search module 104, and the unknown task pre-processing module 105 may include at least a computing device, such as a server. Alternatively, the unknown task processing module 103, the unknown task strategy search module 104, and the unknown task pre-processing module 105 may also be implemented through an application-specific integrated circuit (ASIC) or a programmable logic device (PLD).

[0236] In some possible implementations, the unknown task strategy search module 104 is specifically configured to:

[0237] Processing the original samples in the validation set using a sample processing strategy to obtain enhanced samples, wherein the original samples include the second sample in the unknown task sample;

[0238] A preprocessing strategy is obtained based on the inference results of the initial task model on the original sample and the enhanced sample, wherein the preprocessing strategy includes a sample processing strategy in which a second performance value of the inference on the enhanced sample is greater than a first performance value of the inference on the original sample, and the performance value is an indicator value of a performance evaluation indicator determined according to the task type.

[0239] In some possible implementations, the unknown task strategy search module 104 is further configured to:

[0240] According to the post-processing method corresponding to the task type, the inference results of the original sample and the enhanced sample are fused, and the fused inference results are used to determine the second performance value.

[0241] In some possible implementations, the unknown task strategy search module 104 is further configured to:

[0242] Processing the samples in the test set according to the sample processing strategy to obtain processed test samples;

[0243] The strategy reliability is determined based on the inference results of the initial task model on the processed test samples and the labeling information of the samples in the test set.

[0244] In some possible implementations, the sample processing strategy is obtained by sampling from a search space, and the interaction module is further configured to:

[0245] receiving the user-defined sample processing strategy;

[0246] The unknown task strategy search module 104 is further configured to:

[0247] Add the user-defined sample processing strategy to the search space.

[0248] This application also provides a computing device 800. As shown in Figure 8, computing device 800 includes a bus 802, a processor 804, a memory 806, and a communication interface 808. Processor 804, memory 806, and communication interface 808 communicate with each other via bus 802. Computing device 800 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in computing device 800.

[0249] Bus 802 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, among others. Buses may be classified as address buses, data buses, control buses, and the like. For ease of illustration, FIG8 illustrates a single bus line, but this does not imply a single bus or type of bus. Bus 802 may include a path for transmitting information between various components of computing device 800 (e.g., memory 806, processor 804, and communication interface 808).

[0250] The processor 804 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0251] The memory 806 may include a volatile memory, such as a random access memory (RAM). The memory 806 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD). The memory 806 stores executable program code, and the processor 804 executes the executable program code to implement the aforementioned task processing method. Specifically, the memory 806 stores instructions for the task processing system 10 to execute the task processing method.

[0252] The communication interface 808 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 800 and other devices or a communication network.

[0253] Embodiments of the present application also provide a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0254] As shown in Figure 9, the computing device cluster includes at least one computing device 800. The memory 806 of one or more computing devices 800 in the computing device cluster may store instructions for executing the task processing method using the same task processing system 10.

[0255] In some possible implementations, one or more computing devices 800 in the computing device cluster may also be used to execute some of the instructions for executing the task processing method of the task processing system 10. In other words, the combination of one or more computing devices 800 may jointly execute the instructions for executing the task processing method of the task processing system 10.

[0256] It should be noted that the memories 806 in different computing devices 800 in the computing device cluster may store different instructions for executing partial functions of the task processing system 10 .

[0257] Figure 10 shows a possible implementation. As shown in Figure 10, computing devices 800A and 800B are connected via a communication interface 808. The memory in computing device 800A stores instructions for executing the functions of the interaction module and the unknown task training module 102. The memory in computing device 800B stores instructions for executing the functions of the unknown task strategy search module 104. Furthermore, computing device 800C can also be connected to computing devices 800A and 800B via a communication interface. The memory in computing device 800C also stores instructions for executing the functions of the unknown task preprocessing module 105 and the unknown task processing module 103. The memory 806 of computing devices 800A and 800B (which may also include computing device 800C) jointly stores instructions for the task processing system 10 to execute the task processing method.

[0258] The connection method between the computing device clusters shown in Figure 10 can be considered to take into account that the task processing method provided by this application requires a large amount of computing power for model training and strategy search. Therefore, it is considered to transfer the functions implemented by the unknown task strategy search module 104 to computing device 800B for execution. For similar reasons, the functions implemented by the unknown task preprocessing module 105 and the unknown task processing module 103 are transferred to computing device 800C for execution.

[0259] It should be understood that the functionality of the computing device 800A shown in FIG10 may also be implemented by multiple computing devices 800. Similarly, the functionality of the computing devices 800B and 800C may also be implemented by multiple computing devices 800.

[0260] In some possible implementations, one or more computing devices in a computing device cluster may be connected via a network. The network may be a wide area network or a local area network, etc. FIG11 shows a possible implementation. As shown in FIG11 , computing devices 800D and 800E are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 806 in the computing device 800D stores instructions for executing the functions of the interaction module and the unknown task training module 102. At the same time, the memory 806 in the computing device 800E stores instructions for executing the functions of the unknown task strategy search module 104. Furthermore, the computing device 800F may also be connected to the computing devices 800D and 800E via a network. The memory in the computing device 800F also stores instructions for executing the functions of the unknown task preprocessing module 105 and the unknown task processing module 103.

[0261] The connection method between the computing device clusters shown in Figure 11 can be considered to take into account that the task processing method provided by this application requires a large amount of computing power for model training and strategy search. Therefore, it is considered to transfer the functions implemented by the unknown task strategy search module 104 to computing device 800E for execution. For similar reasons, the functions implemented by the unknown task preprocessing module 105 and the unknown task processing module 103 are transferred to computing device 800F for execution.

[0262] It should be understood that the functionality of the computing device 800D shown in FIG11 may also be implemented by multiple computing devices 800. Similarly, the functionality of the computing devices 800E and 800F may also be implemented by multiple computing devices 800.

[0263] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the above-mentioned task processing method applied to the task processing system 10.

[0264] The present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored on any available medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to perform the task processing method described above.

[0265] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the protection scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A task processing method, characterized in that: Applied to a task processing system, the task processing system is used to provide a task model for processing unknown tasks, and the unknown tasks are processed based on the task model. The method includes: Receive the initial task model, known task samples, and unknown task samples input by the user; Inputting at least one task sample into the initial task model for reasoning, and determining a weight update strategy according to the reasoning result; The known task samples and the unknown task samples are input into the initial task model, and the model is trained through the weight updating strategy to obtain the task model.

2. The method according to claim 1, characterized in that The known task samples are samples that are successfully inferred by the initial task model, and the unknown task samples are samples that fail to be inferred by the initial task model.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: Determine a performance evaluation indicator according to the reasoning result; The step of inputting the known task samples and the unknown task samples into the initial task model, performing model training through the weight updating strategy, and obtaining the task model comprises: According to the samples in the training set, iteratively updating the weight of the initial task model through the weight update strategy to obtain a transient model, wherein the training set includes the first sample in the unknown task sample and the known task sample; According to the samples in the validation set, the transient model is subjected to performance evaluation by the performance evaluation index to obtain a performance value, wherein the validation set includes the second sample in the unknown task sample; When the performance value of the transient model reaches a performance threshold or the number of iterations of the initial task model reaches a maximum number, the transient model is output as the task model.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: Determine a model reliability evaluation index according to the inference result; According to the samples in the test set, the reliability of the task model is determined by using the model reliability evaluation index.

5. The method according to any one of claims 1 to 4, characterized in that: Determining the weight update strategy according to the inference result includes: Determine the task type according to the inference result, and determine the weight update strategy corresponding to the task type; The task type includes a classification task, a detection task or a segmentation task. The weight update strategy corresponding to the classification task includes a sample-level update strategy, the weight update strategy corresponding to the detection task includes an instance-level update strategy, and the weight update strategy corresponding to the segmentation task includes a pixel-level update strategy.

6. The method according to claim 5, characterized in that The performance evaluation indicators corresponding to the classification task include the F value, the model reliability evaluation indicators corresponding to the classification task include the sample classification error rate, the performance evaluation indicators corresponding to the detection task include the mean average precision mAP, the model reliability evaluation indicators corresponding to the detection task include the instance regression error rate and the instance classification error rate, the performance evaluation indicators corresponding to the segmentation task include the average intersection-over-union ratio mIOU, and the model reliability evaluation indicators corresponding to the segmentation task include the pixel classification error rate.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Acquire a preprocessing strategy according to the unknown task sample, the initial task model and the task type determined by the reasoning result; Using the preprocessing strategy to enhance the unknown task to be processed; The enhanced unknown task is processed by the task model.

8. The method according to claim 7, characterized in that The obtaining of a preprocessing strategy according to the unknown task sample, the initial task model and the task type determined by the reasoning result includes: Processing the original samples in the validation set by using a sample processing strategy to obtain enhanced samples, wherein the original samples include the second samples in the unknown task samples; A preprocessing strategy is obtained according to the inference results of the initial task model on the original sample and the enhanced sample, wherein the preprocessing strategy includes a sample processing strategy in which a second performance value of the inference on the enhanced sample is greater than a first performance value of the inference on the original sample, and the performance value is an indicator value of a performance evaluation indicator determined according to the task type.

9. The method according to claim 8, characterized in that The method further comprises: According to the post-processing method corresponding to the task type, the reasoning results of the original sample and the enhanced sample are fused, and the fused reasoning results are used to determine the second performance value.

10. The method according to claim 8, characterized in that The method further comprises: According to the sample processing strategy, the samples in the test set are processed to obtain processed test samples; The strategy reliability is determined according to the inference result of the initial task model on the processed test samples and the labeling information of the samples in the test set.

11. The method according to any one of claims 8 to 10, characterized in that: The sample processing strategy is obtained by sampling from the search space, and the method further includes: Receiving the user-defined sample processing strategy; The user-defined sample processing strategy is added to the search space.

12. A task processing system, characterized in that: The task processing system is used to provide a task model for processing unknown tasks, and processes the unknown tasks based on the task model. The system includes: An interactive module, used to receive the initial task model, known task samples, and unknown task samples input by the user; The unknown task training module is used to input at least one task sample into the initial task model for reasoning, determine the weight update strategy according to the reasoning result, input the known task sample and the unknown task sample into the initial task model, perform model training through the weight update strategy, and obtain the task model.

13. The system according to claim 12, characterized in that The known task samples are samples that are successfully inferred by the initial task model, and the unknown task samples are samples that fail to be inferred by the initial task model.

14. The system according to claim 12 or 13, characterized in that The unknown task training module is also used for: Determine a performance evaluation indicator according to the reasoning result; The unknown task training module is specifically used for: According to the samples in the training set, iteratively updating the weight of the initial task model through the weight update strategy to obtain a transient model, wherein the training set includes the first sample in the unknown task sample and the known task sample; According to the samples in the validation set, the transient model is subjected to performance evaluation by the performance evaluation index to obtain a performance value, wherein the validation set includes the second sample in the unknown task sample; When the performance value of the transient model reaches a performance threshold or the number of iterations of the initial task model reaches a maximum number, the transient model is output as the task model.

15. The system according to any one of claims 12 to 14, characterized in that: The unknown task training module is also used for: Determine a model reliability evaluation index according to the inference result; According to the samples in the test set, the reliability of the task model is determined by using the model reliability evaluation index.

16. The system according to any one of claims 12 to 15, characterized in that The unknown task training module is specifically used for: Determine the task type according to the inference result, and determine the weight update strategy corresponding to the task type; The task type includes a classification task, a detection task or a segmentation task. The weight update strategy corresponding to the classification task includes a sample-level update strategy, the weight update strategy corresponding to the detection task includes an instance-level update strategy, and the weight update strategy corresponding to the segmentation task includes a pixel-level update strategy.

17. The system according to claim 16, characterized in that The performance evaluation indicators corresponding to the classification task include the F value, the model reliability evaluation indicators corresponding to the classification task include the sample classification error rate, the performance evaluation indicators corresponding to the detection task include the mean average precision mAP, the model reliability evaluation indicators corresponding to the detection task include the instance regression error rate and the instance classification error rate, the performance evaluation indicators corresponding to the segmentation task include the average intersection-over-union ratio mIOU, and the model reliability evaluation indicators corresponding to the segmentation task include the pixel classification error rate.

18. The system according to any one of claims 12 to 17, characterized in that The system further comprises: An unknown task strategy search module, used to obtain a preprocessing strategy according to the unknown task sample, the initial task model and the task type determined by the reasoning result; An unknown task preprocessing module, used for enhancing the unknown task to be processed using the preprocessing strategy; The unknown task processing module is used to process the enhanced unknown task through the task model.

19. The system according to claim 18, characterized in that The unknown task strategy search module is specifically used for: Processing the original samples in the validation set by using a sample processing strategy to obtain enhanced samples, wherein the original samples include the second samples in the unknown task samples; A preprocessing strategy is obtained according to the inference results of the initial task model on the original sample and the enhanced sample, wherein the preprocessing strategy includes a sample processing strategy in which a second performance value of the inference on the enhanced sample is greater than a first performance value of the inference on the original sample, and the performance value is an indicator value of a performance evaluation indicator determined according to the task type.

20. The system according to claim 19, characterized in that The unknown task strategy search module is also used for: According to the post-processing method corresponding to the task type, the reasoning results of the original sample and the enhanced sample are fused, and the fused reasoning results are used to determine the second performance value.

21. The system according to claim 19, characterized in that The unknown task strategy search module is also used for: According to the sample processing strategy, the samples in the test set are processed to obtain processed test samples; The strategy reliability is determined according to the inference result of the initial task model on the processed test samples and the labeling information of the samples in the test set.

22. The system according to any one of claims 19 to 21, characterized in that The sample processing strategy is obtained by sampling from the search space, and the interaction module is further used for: Receiving the user-defined sample processing strategy; The unknown task strategy search module is also used for: The user-defined sample processing strategy is added to the search space.

23. A computing device cluster, characterized in that: The computing device cluster includes at least one computing device, and the at least one computing device includes at least one processor and at least one memory, wherein the at least one memory stores computer-readable instructions; the at least one processor executes the computer-readable instructions so that the computing device cluster executes the method as described in any one of claims 1 to 11.

24. A computer-readable storage medium, characterized in that: The method comprises computer-readable instructions; the computer-readable instructions are used to implement the method according to any one of claims 1 to 11.

Citation Information

Patent Citations

  • Multi-task model training method and device and multi-task processing method and device

    CN114202026A

  • Task learning system and method and related equipment

    CN114444696A

  • Model training method and related device

    CN115423031A

  • Method for constructing AI integrated model and reasoning method and device of AI integrated model

    CN115964632A

  • Neural network training method, data processing method, and related apparatus

    US20220215259A1

Cited By

  • Model performance evaluation method and device, electronic equipment and medium

    CN121190915A

  • A method, apparatus, electronic device, and medium for evaluating the performance of a remote sensing model.

    CN121190915B