Task processing method and related system

By identifying the output of the initial task model and adapting the weight update strategy, the model performance degradation and coupling problems in unknown task processing are solved, and efficient and pluggable unknown task processing is achieved, reducing costs and improving adaptability.

CN118917432APending Publication Date: 2024-11-08HUAWEI CLOUD COMPUTING TECHNOLOGIES CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202311264690.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-05-06
Filing Date
2023-09-27
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

When the prior art deals with unknown tasks, the model performance drops sharply, resulting in unpredictable accidents and economic losses. The unknown task processing method is highly coupled with specific task models, making it difficult to achieve pluggable model and cannot meet business needs.

Method used

By identifying the output of the initial task model and adapting the weight update strategy, any type of initial task model is implemented as input for unknown task training based on instance re-empowerment, decoupling the processing logic and code levels, and has the characteristics of pluggable models, and Accelerate the training process by performance evaluation metrics and reliability evaluation.

Benefits of technology

It realizes efficient processing of unknown tasks, reduces labor and time costs, improves the adaptability and deployment of the model, and can quickly obtain a task model with effective processing capabilities for unknown tasks to meet business needs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118917432A_ABST
    Figure CN118917432A_ABST
Patent Text Reader

Abstract

The invention provides a task processing method, which is applied to a task processing system, the task processing system is used for providing a task model for processing an unknown task and processing the unknown task based on the task model, and the method comprises the following steps: receiving an initial task model, a known task sample and an unknown task sample input by a user, and inputting the at least one task sample into the initial task model for reasoning, determining a weight updating strategy according to a reasoning result, inputting the known task sample and the unknown task sample into the initial task model, and performing model training through the weight updating strategy to obtain a task model. According to the method, a weight updating strategy is adapted to a reasoning result of a task sample through an initial task model, so that unknown task training based on instance re-empowerment can be carried out by taking any type of initial task model as input, and the method has the characteristic that the model is pluggable.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] This application claims the priority of the Chinese patent application filed with the State Intellectual Property Office of China on May 6, 2023, with application number 202310505723.6 and invention name “A task processing method and related system”, the entire contents of which are incorporated by reference in this application. Technical Field

[0002] The present application relates to the field of artificial intelligence (AI) technology, and in particular to a task processing method, system, computing device cluster, computer-readable storage medium, and computer program product. Background Art

[0003] In recent years, data-driven artificial intelligence (AI) technology has exploded with vitality in various fields. Deep learning models based on supervised learning (also referred to as task models or models) can achieve good performance by processing samples within the distribution of training data (also called known tasks). However, in the real world, the model may encounter data outside the distribution of training data (also called unknown tasks) at any time, causing the model performance to drop sharply, thereby causing unpredictable accidents and losses. For example, in industrial quality inspection, the detection performance of the model can drop by up to 80% due to changes in factors such as raw materials, production processes, and quality inspection environment. This situation may not only lead to the failure to deliver industrial products on time, causing huge economic losses, but in serious cases, it may even cause safety accidents on industrial production lines.

[0004] For unknown tasks, it is usually necessary to collect a large amount of unknown sample data and manually annotate it, then take the model offline from the business system, and finally retrain the model to adapt to the unknown task. The collection and annotation of a large amount of unknown sample data brings huge human resource costs and time costs, and the stagnation of the business system will also cause huge economic costs.

[0005] To this end, the industry has proposed some unknown task processing methods, such as adjusting the weights of known and unknown tasks during the training process to improve the task model's ability to handle unknown tasks. However, the above unknown task processing methods are usually highly coupled with the specific task model in terms of processing logic and code. Once the task model changes or the task model no longer has a certain structure, these unknown task processing methods will become invalid and difficult to meet business needs. Summary of the invention

[0006] The present application provides a task processing method, which can realize unknown task training based on instance re-weighting with any type of initial task model as input by identifying the output of the initial task model (for example, the inference result of the initial task model on the task sample) and adapting the weight update strategy, and has the feature of pluggable model. The present application also provides a task processing system, a computing device cluster, a computer-readable storage medium, and a computer program product corresponding to the above method.

[0007] In a first aspect, the present application provides a task processing method. The method is applied to a task processing system. The task processing system is used to provide a task model for processing unknown tasks, and to process unknown tasks based on the task model. The task processing system can be a software system, which can be deployed in a computing device cluster, for example, in a cloud computing cluster provided by a cloud service provider, or in an edge computing cluster. The computing device cluster executes the program code of the software system to execute the task processing method of the present application. In some possible implementations, the task processing system can also be a hardware system, for example, a computing device cluster with an unknown task processing function. When the hardware system is running, the task processing method of the present application is executed.

[0008] Specifically, the task processing system can receive the initial task model, known task samples, and unknown task samples input by the user, input at least one task sample into the initial task model for reasoning, determine the weight update strategy based on the reasoning results, and then input the known task samples and the unknown task samples into the initial task model, perform model training through the weight update strategy, and obtain the task model.

[0009] This method identifies the output of the initial task model (for example, the reasoning result of the initial task model on the task sample) and adapts the weight updating strategy, so that it can use any type of initial task model as input to perform instance-based re-weighting unknown task training. The unknown task processing method is decoupled from the task model (or model structure) at the processing logic and code levels. Even if the task model changes or the task model no longer has a certain structure, a task model that is effective for unknown task processing can be trained, and the model is pluggable.

[0010] In some possible implementations, known task samples are samples that are successfully inferred by the initial task model, and unknown task samples are samples that fail to be inferred by the initial task model. The initial task model can be a business model offline from the business system, which is used for retraining to adapt to unknown tasks. This method uses the known task samples that are successfully inferred by the initial task model and the unknown task samples that fail to be inferred to retrain the initial task model, so as to quickly obtain a task model that has effective processing capabilities for unknown tasks and meet business needs.

[0011] In some possible implementations, the task processing system may also determine a performance evaluation index based on the inference result. The task processing system may iteratively update the weight of the initial task model through a weight update strategy based on samples in the training set to obtain a transient model, and perform a performance evaluation on the transient model through the performance evaluation index based on samples in the validation set to obtain a performance value, wherein the training set includes a first sample in an unknown task sample and a known task sample, and the validation set includes a second sample in an unknown task sample. When the performance value of the transient model reaches a performance threshold or the number of iterations of the initial task model reaches a maximum number, the task processing system outputs the transient model as the task model.

[0012] This method also adapts performance evaluation indicators based on the inference results. Appropriate performance evaluation indicators can speed up the training efficiency of the task model and enable a task model that meets the requirements to be obtained more quickly.

[0013] In some possible implementations, the task processing system can also determine the model reliability evaluation index based on the inference results, and then determine the reliability of the task model based on the samples in the test set through the model reliability evaluation index. In this way, the user can be fed back the unknown task processing status. When the reliability of the task model is low, the user can be fed back to collect and annotate more unknown task samples to train a task model that is sufficiently effective for the current unknown task processing.

[0014] In some possible implementations, the task processing system can determine the task type according to the inference result and determine the weight update strategy corresponding to the task type. The task type includes a classification task, a detection task, or a segmentation task. The weight update strategy corresponding to the classification task includes a sample-level update strategy, the weight update strategy corresponding to the detection task includes an instance-level update strategy, and the weight update strategy corresponding to the segmentation task includes a pixel-level update strategy.

[0015] This method identifies the task type based on the inference results and adapts the corresponding weight update strategy according to the task type. Using this weight update strategy to update the model weights can make the model converge faster, thereby accelerating training efficiency.

[0016] In some possible implementations, the performance evaluation indicators corresponding to the classification task include the F value, the model reliability evaluation indicators corresponding to the classification task include the sample classification error rate, the performance evaluation indicators corresponding to the detection task include the mean average precision mAP, the model reliability evaluation indicators corresponding to the detection task include the instance regression error rate and the instance classification error rate, the performance evaluation indicators corresponding to the segmentation task include the average intersection-over-union ratio mIOU, and the model reliability evaluation indicators corresponding to the segmentation task include the pixel classification error rate.

[0017] This method can provide richer reference information for model training by adapting the corresponding performance evaluation indicators and model reliability evaluation indicators to the task type.

[0018] In some possible implementations, the task processing system may also obtain a preprocessing strategy based on the unknown task sample, the initial task model, and the task type determined by the reasoning result, and then use the preprocessing strategy to enhance the unknown task to be processed, and process the enhanced unknown task through the task model.

[0019] This method can use any type of task model as input to search for strategies in the reasoning phase, and has the feature of pluggable models. In addition, the introduction of preprocessing strategies for data enhancement in the task model reasoning phase can improve the detection performance of the task model for unknown tasks in real time. In addition, both unknown task training and unknown task strategy search achieve model-independent unknown task processing, which improves the deployability and versatility of the entire unknown task processing system.

[0020] In some possible implementations, the task processing system may process the original samples in the validation set through a sample processing strategy to obtain an enhanced sample, the original sample including the second sample in the unknown task sample, and then the task processing system may obtain a preprocessing strategy based on the inference results of the original sample and the enhanced sample by the initial task model. The preprocessing strategy includes a sample processing strategy in which a second performance value of reasoning about the enhanced sample is greater than a first performance value of reasoning about the original sample, and the performance value is an indicator value of a performance evaluation indicator determined according to the task type.

[0021] This method introduces an external enhancement library (such as a library that provides sample processing strategies) and uses optimization methods to search for an effective set of data enhancement strategies and enhancement strengths for the task model, thereby improving the gain effect of data enhancement on the task model.

[0022] In some possible implementations, the task processing system may also fuse the reasoning results of the original sample and the enhanced sample according to the post-processing method corresponding to the task type, and the fused reasoning results are used to determine the second performance value. The fused reasoning results have higher reliability, and the performance values ​​determined based on the fused reasoning results are also more accurate, thereby improving the accuracy of strategy search. It should be noted that the present application also supports the integration of the results of multiple enhanced samples by voting and other methods, as a supplement to the introduced enhancement method, which can effectively filter out redundant results.

[0023] In some possible implementations, the task processing system may also process samples in the test set according to the sample processing strategy to obtain processed test samples, and determine the strategy reliability based on the inference results of the initial task model on the processed test samples and the labeling information of the samples in the test set.

[0024] This method uses an additional output strategy reliability to indicate the confidence of the searched strategy in processing unknown tasks. In this way, when the reliability is low, the user can be fed back to input more sample processing strategies to search for preprocessing strategies that are sufficiently effective for unknown tasks.

[0025] In some possible implementations, the sample processing strategy is sampled from the search space, and the task processing system can also receive the user-defined sample processing strategy and add the user-defined sample processing strategy to the search space, thereby expanding the strategy search capability and having high flexibility.

[0026] In a second aspect, the present application provides a task processing system. The task processing system is used to provide a task model for processing unknown tasks, and to process unknown tasks based on the task model. The system includes:

[0027] An interactive module, used to receive the initial task model, known task samples, and unknown task samples input by the user;

[0028] The unknown task training module is used to input at least one task sample into the initial task model for reasoning, determine the weight update strategy according to the reasoning result, input the known task sample and the unknown task sample into the initial task model, perform model training through the weight update strategy, and obtain the task model.

[0029] In some possible implementations, the known task samples are samples that are successfully inferred by the initial task model, and the unknown task samples are samples that are failed to be inferred by the initial task model.

[0030] In some possible implementations, the unknown task training module is further used to:

[0031] Determine a performance evaluation indicator according to the reasoning result;

[0032] The unknown task training module is specifically used for:

[0033] According to the samples in the training set, iteratively updating the weight of the initial task model through the weight update strategy to obtain a transient model, wherein the training set includes the first sample in the unknown task sample and the known task sample;

[0034] According to the samples in the validation set, the transient model is subjected to performance evaluation by the performance evaluation index to obtain a performance value, wherein the validation set includes the second sample in the unknown task sample;

[0035] When the performance value of the transient model reaches a performance threshold or the number of iterations of the initial task model reaches a maximum number, the transient model is output as the task model.

[0036] In some possible implementations, the unknown task training module is further used to:

[0037] Determine a model reliability evaluation index according to the inference result;

[0038] According to the samples in the test set, the reliability of the task model is determined by using the model reliability evaluation index.

[0039] In some possible implementations, the unknown task training module is specifically used to:

[0040] Determine the task type according to the inference result, and determine the weight update strategy corresponding to the task type;

[0041] The task type includes a classification task, a detection task or a segmentation task. The weight update strategy corresponding to the classification task includes a sample-level update strategy, the weight update strategy corresponding to the detection task includes an instance-level update strategy, and the weight update strategy corresponding to the segmentation task includes a pixel-level update strategy.

[0042] In some possible implementations, the performance evaluation indicators corresponding to the classification task include the F value, the model reliability evaluation indicators corresponding to the classification task include the sample classification error rate, the performance evaluation indicators corresponding to the detection task include the mean average precision mAP, the model reliability evaluation indicators corresponding to the detection task include the instance regression error rate and the instance classification error rate, the performance evaluation indicators corresponding to the segmentation task include the average intersection-over-union ratio mIOU, and the model reliability evaluation indicators corresponding to the segmentation task include the pixel classification error rate.

[0043] In some possible implementations, the system further includes:

[0044] An unknown task strategy search module, used to obtain a preprocessing strategy according to the unknown task sample, the initial task model and the task type determined by the reasoning result;

[0045] An unknown task preprocessing module, used for enhancing the unknown task to be processed using the preprocessing strategy;

[0046] The unknown task processing module is used to process the enhanced unknown task through the task model.

[0047] In some possible implementations, the unknown task strategy search module is specifically used to:

[0048] Processing the original samples in the validation set by using a sample processing strategy to obtain enhanced samples, wherein the original samples include the second samples in the unknown task samples;

[0049] A preprocessing strategy is obtained according to the inference results of the initial task model on the original sample and the enhanced sample, wherein the preprocessing strategy includes a sample processing strategy in which a second performance value of the inference on the enhanced sample is greater than a first performance value of the inference on the original sample, and the performance value is an indicator value of a performance evaluation indicator determined according to the task type.

[0050] In some possible implementations, the unknown task strategy search module is further used to:

[0051] According to the post-processing method corresponding to the task type, the reasoning results of the original sample and the enhanced sample are fused, and the fused reasoning results are used to determine the second performance value.

[0052] In some possible implementations, the unknown task strategy search module is further used to:

[0053] According to the sample processing strategy, the samples in the test set are processed to obtain processed test samples;

[0054] The strategy reliability is determined according to the inference result of the initial task model on the processed test samples and the labeling information of the samples in the test set.

[0055] In some possible implementations, the sample processing strategy is obtained by sampling from a search space, and the interaction module is further used to:

[0056] Receiving the user-defined sample processing strategy;

[0057] The unknown task strategy search module is also used for:

[0058] The user-defined sample processing strategy is added to the search space.

[0059] In a third aspect, the present application provides a computing device cluster. The computing device cluster includes at least one computing device, and the at least one computing device includes at least one processor and at least one memory. The at least one processor and the at least one memory communicate with each other. The at least one processor is used to execute instructions stored in the at least one memory, so that the computing device or the computing device cluster performs the task processing method described in the first aspect or any implementation of the first aspect.

[0060] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, wherein the instructions instruct a computing device or a computing device cluster to execute the task processing method described in the first aspect or any one of the implementations of the first aspect.

[0061] In a fifth aspect, the present application provides a computer program product comprising instructions, which, when executed on a computing device or a computing device cluster, enables the computing device or the computing device cluster to execute the task processing method described in the first aspect or any one of the implementations of the first aspect.

[0062] Based on the implementations provided in the above aspects, this application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical method of the embodiments of the present application, the drawings required for use in the embodiments are briefly introduced below.

[0064] Figure 1 A schematic diagram of the architecture of a task processing system provided in an embodiment of the present application;

[0065] Figure 2 A schematic diagram of a deployment form of a task processing system provided in an embodiment of the present application;

[0066] Figure 3 A flowchart of a task processing method provided in an embodiment of the present application;

[0067] Figure 4 A schematic diagram of a process of training a task model for an unknown task training module provided in an embodiment of the present application;

[0068] Figure 5 A schematic diagram of a flow chart of an unknown task strategy search module searching for a preprocessing strategy provided in an embodiment of the present application;

[0069] Figure 6 A schematic diagram of a scenario in which a task processing method provided in an embodiment of the present application is applied to a visual inspection task of surface defects of industrial products;

[0070] Figure 7 A schematic diagram of a scenario in which a task processing method improved in an embodiment of the present application is applied to an autonomous driving task or a visual inspection task for surface defects of industrial products;

[0071] Figure 8 A schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0072] Fig. 9A schematic diagram of a computing device cluster provided in an embodiment of the present application;

[0073] Fig.10 Another schematic diagram of a computing device cluster provided in an embodiment of the application;

[0074] Fig.11 A schematic diagram of the structure of another computer cluster provided in an embodiment of the application. DETAILED DESCRIPTION

[0075] The terms "first" and "second" in the embodiments of the present application are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.

[0076] First, some technical terms involved in the embodiments of the present application are introduced.

[0077] In the field of artificial intelligence (AI), a task model refers to a data-driven model used to process tasks to achieve corresponding purposes or functions. Specifically, a task model can be a deep learning model based on supervised learning. In some examples, the task model can be a classification model, a target detection model, or a segmentation model trained based on supervised learning.

[0078] Unknown tasks, also known as unknown task data or unknown data, specifically refer to data outside the knowledge distribution of the task model. Known tasks, also known as known task data or known data, specifically refer to data within the knowledge distribution of the task model. The knowledge distribution may be the data distribution of the task model learning, such as the training data distribution.

[0079] Task models can usually achieve good performance for known tasks, but their performance drops sharply when processing unknown tasks. If task models are used directly to process unknown tasks, it may cause unpredictable accidents and losses. For example, in industrial quality inspection, the detection performance of task models may drop by up to 80% due to changes in raw materials, production processes, quality inspection environment and other factors, which may not only lead to failure to deliver products on time and cause economic losses, but may also cause safety accidents in industrial production lines.

[0080] Considering that collecting and labeling a large number of unknown task samples, and retraining the task model based on the labeled unknown task samples to adapt to the unknown tasks requires huge human resource costs and time costs, the industry has proposed some low-cost unknown task processing methods.

[0081] A typical unknown task processing method is the unknown task processing method based on instance weighting. This method improves the task model's ability to process unknown data by adjusting the weights of known tasks and unknown tasks during the training process. In addition, this method can also alleviate the model's dependence on the amount of unknown task data to a certain extent, reducing the workload of collecting and annotating data.

[0082] Another typical unknown task processing method is the unknown task processing method based on the enhancement in the reasoning stage. This method applies different data enhancement transformations to unknown tasks in the reasoning stage, and improves the task model's processing ability for unknown tasks by integrating the idea of ​​reasoning. This method does not require retraining the model, avoiding the human resource cost and time cost caused by retraining the model.

[0083] The above-mentioned unknown task processing methods (unknown task processing methods based on instance empowerment and unknown task processing methods based on reasoning stage enhancement) are usually highly coupled with specific task models in terms of processing logic and code, making it difficult to achieve model pluggability. Model pluggability means that for a certain algorithm or technology, it can use any task model as input, execute the algorithm or technical process independently of the model, and optimize the weight of the task model or the reasoning result of the task model.

[0084] Once the task model changes or the task model no longer has a certain structure, these non-model-independent technologies will become invalid or need to be readjusted. For example, in the process of multi-party collaborative development, once the business model is switched, or multiple parties use different business models, the above-mentioned unknown task processing methods will become invalid or need to be readjusted because the model cannot be pluggable. For example, in the multi-party collaborative development process of A, B, and C, C provides A and B with an unknown task processing solution, and A and B both have their own task models. If C's solution is coupled to a certain model, it will result in either C developing a set of unknown task processing methods for A and B's task models respectively, or A and B will need to adjust their own task models, making it difficult for multiple parties to work together.

[0085] In view of this, the present application provides a task processing method. The method can be executed by a task processing system. The task processing system is used to provide a task model for processing unknown tasks, and to process unknown tasks based on the task model. The task processing system can be a software system, and the software system can be deployed in a computing device cluster, for example, in a cloud computing cluster provided by a cloud service provider, or in an edge computing cluster. The computing device cluster executes the program code of the software system to execute the task processing method of the present application. In some possible implementations, the task processing system can also be a hardware system, for example, a computing device cluster with an unknown task processing function. When the hardware system is running, the task processing method of the present application is executed.

[0086] Specifically, the task processing system can receive the initial task model, known task samples, and unknown task samples input by the user, input at least one task sample into the initial task model for reasoning, determine the weight update strategy based on the reasoning results, and then input the known task samples and the unknown task samples into the initial task model, perform model training through the weight update strategy, and obtain the task model for unknown task processing.

[0087] This method recognizes the output of the initial task model (for example, the inference result of the initial task model on the task sample) and adapts the weight update strategy, so as to realize unknown task training based on instance re-weighting with any type of initial task model as input, and has the feature of pluggable model. Furthermore, this method can also adapt the model reliability evaluation index based on the inference result of the initial task model on the task sample, and accordingly, the reliability of the task model can be output based on the model reliability evaluation index, so as to provide feedback to the user on the unknown task processing status. When the reliability of the task model is low, the user can be fed back to collect and annotate more unknown task samples to train a task model that is effective enough for the current unknown task processing.

[0088] In addition, the method also supports the acquisition of preprocessing strategies, which can be based on the inference results of the task samples of the initial task model, adapted to the performance evaluation indicators corresponding to the task type, and further adapted to the post-processing method corresponding to the task type, so as to realize the strategy search in the reasoning stage with any type of task model as input, and has the feature of pluggable model. Similarly, the method can also adapt the strategy reliability evaluation index, and accordingly, the reliability of the preprocessing strategy can also be output based on the strategy reliability evaluation index, so as to provide feedback to the user. In addition, the method supports user-defined sample processing strategies, and adds user-defined sample processing strategies to the search space for strategy search, which can expand the unknown task processing capabilities.

[0089] In order to make the technical solution of the present application clearer and easier to understand, the system architecture of the present application is introduced below with reference to the accompanying drawings.

[0090] See also Figure 1 The task processing system 10 includes an interaction module ( Figure 1) and unknown task training module 102. Further, the task processing system 10 may also include an unknown task processing module 103, an unknown task strategy search module 104, and an unknown task preprocessing module 105. Accordingly, the above modules are functional modules. The functional module may be a software module. For example, when the task processing system 10 is an integrated software system, the interactive module is a module that provides interaction, which may generally be a front-end module, such as a user interface (UI) module. The unknown task training module 102, the unknown task processing module 103, the unknown task strategy search module 104, and the unknown task preprocessing module 105 may be code blocks that can implement corresponding business logic. For another example, when the task processing system 10 is a distributed software system, the interactive module, the unknown task training module 102, and the unknown task strategy search module 104 may be microservices that can implement corresponding functions. The functional module may also be a hardware module, such as a processor in a computing device, or a computing device (such as a server) in a computing device cluster.

[0091] The input of the task processing system 10 may include an initial task model and training data, wherein the training data includes known task samples and unknown task samples. The task processing system 10 also includes unknown tasks to be processed, also known as unknown task reasoning data. In some possible implementations, the input of the task processing system 10 may also include a user-defined sample processing strategy. The following is an introduction to each of the inputs of the task processing system 10:

[0092] Input 1: Initial task model. The initial task model is specifically a task model to be trained, and the trained task model is used to process the business. The initial task model can be a classification model, target detection model, segmentation model, or other machine learning model based on deep learning. In some implementations, the initial task model can be a business model offline from the business system, which is used for retraining to adapt to unknown tasks.

[0093] Input 2: training data. The training data can be a collection of known task samples and unknown task samples with labeled information. The known task samples are samples where the initial task model is successfully inferred (such as correct inference), and the unknown task samples are samples where the initial task model fails to infer (such as inference errors or inability to provide inference results).

[0094] Input 3: unknown task reasoning data. Unknown task reasoning data can be abnormal distribution data (data outside the knowledge distribution of the task model) encountered during the online operation of the task model, that is, unknown tasks to be processed.

[0095] Input 4: User-defined sample processing strategy. Users can customize one or more sample processing strategies, which can be organized in a list. Each element in the list can be a tuple, representing a user-defined sample processing function and the parameter value range of the function. It should be noted that the task processing method of the present application can also be executed without inputting a user-defined sample processing strategy. For example, a strategy search can be performed based on the system's built-in sample processing strategy.

[0096] The output of the task processing system 10 may include a task model and an unknown task reasoning result (for example, the result of processing an unknown task using a trained task model). In some possible implementations, the output of the task processing system 10 may also include one or more of the reliability of the task model, the preprocessing strategy, and the reliability of the preprocessing strategy. The outputs of the task processing system 10 are introduced below:

[0097] Output 1: Task model. The task model includes model weights trained by the unknown task training module 102 and can adapt to the data distribution of the unknown task.

[0098] Output 2: unknown task reasoning result. The unknown task reasoning result is usually related to the task (or task model). For example, if the input task model is a classification model or the task is a classification task, the unknown task reasoning result can be the probability that the image belongs to each class.

[0099] Output 3: Reliability of the task model. The reliability of the task model can be characterized by the confidence of the model weight output by the unknown task training module 102 for unknown task processing. This output is related to the recognition result of the task model by the task processing system 10. For example, if the recognition result is a classification model, the reliability of the task model can be the classification error rate of the task model for unknown task samples in the test set. For another example, if the recognition result is a target detection model, the reliability of the task model can be the classification and regression error rate of the task model for each label box of the unknown task samples in the task test set. If the reliability is low, more unknown task samples provided and labeled by the user can be fed back to the task processing system 10 to train a sufficiently effective task model.

[0100] Output 4: preprocessing strategy. The preprocessing strategy can be a sample processing strategy that is helpful for unknown task reasoning and is searched by the unknown task strategy search module 104. The sample processing strategy can include a sample enhancement method (which can be represented by a sample processing function) and strength (which can be represented by parameters of the sample processing function). The preprocessing strategy can be a list, each element in the list is a sublist, and each element in the sublist is a two-tuple, representing a specific sample processing function and strength.

[0101] Output 5: Reliability of the preprocessing strategy. The reliability of the preprocessing strategy can be characterized by the confidence of the preprocessing strategy output by the unknown task strategy search module 104, and this output is related to the recognition result of the task model by the task processing system 10. For example, if the recognition result is a classification model, the reliability of the preprocessing strategy can be the classification error rate of the unknown task samples in the test set inferred by the task model using the preprocessing strategy. When the reliability is low, the user can be fed back to provide a customized sample processing strategy to expand the search space in order to search for a preprocessing strategy that is effective enough for the unknown task to be processed.

[0102] Next, the modules in the task processing system 10 , such as the interaction module, the unknown task training module 102 , the unknown task processing module 103 , the unknown task strategy search module 104 , and the unknown task preprocessing module 105 , are introduced.

[0103] The interaction module is used to receive the initial task model, known task samples, and unknown task samples input by the user. Specifically, the interaction module can provide an interaction interface to the user. The interaction interface can be a graphical user interface (GUI) or a command user interface (CUI). The user can input the initial task model, known task samples, and unknown task samples through an interaction interface such as a GUI or CUI. For example, the interaction interface can display a model list, and the user can select a model from the model list displayed on the interaction interface. The model selected by the user can be regarded as the initial task model input by the user. The user can enter the path of the training data in the interaction interface, and the interaction module can obtain the training data according to the path entered by the user. The training data can be a collection of known task samples and unknown task samples.

[0104] The unknown task training module 102 is used to input at least one task sample into the initial task model for reasoning, determine the weight update strategy according to the reasoning result, input the known task sample and the unknown task sample into the initial task model, perform model training through the weight update strategy, and obtain the task model. The unknown task processing module 103 is used to process the unknown task to be processed through the task model.

[0105] Furthermore, the unknown task strategy search module 104 can also be used to obtain a preprocessing strategy based on the unknown task sample, the initial task model and the task type determined by the reasoning result. The unknown task preprocessing module 105 is used to enhance the unknown task to be processed using the preprocessing strategy. Accordingly, the unknown task processing module 103 is specifically used to process the enhanced unknown task through the task model.

[0106] like Figure 1As shown in Figure 1, unknown task processing can be divided into a training phase and an inference phase. The training phase and the inference phase are described below.

[0107] In the training phase, the unknown task training module 102 performs model training according to the initial task model and training data (including known task samples and unknown task samples) input by the user to obtain a task model. Furthermore, the unknown task training module 102 can also output the reliability of the task model. The unknown task strategy search module 10 can search for a preprocessing strategy for the unknown task to be processed based on the initial task model and training data, such as the unknown task samples in the training data. Furthermore, the unknown task strategy search module 104 can also output the reliability of the preprocessing strategy.

[0108] In the inference stage, the unknown task processing module 103 processes the unknown task to be processed (unknown task inference data) through the task model output by the unknown task training module 102. Among them, using the task model to process the unknown task to be processed can be using the task model to perform inference and output the unknown task inference result. Figure 1 In the example, the unknown task preprocessing module 105 can first enhance the unknown task to be processed through the preprocessing strategy output by the unknown task strategy search module 104, and accordingly, the unknown task processing module 103 processes the enhanced unknown task through the task model output by the unknown task training module 102.

[0109] Figure 1 The architecture of the task processing system 10 of the embodiment of the present application is introduced. The deployment form of the task processing system 10 is illustrated below. In some examples, the above-mentioned task processing system 10 can be deployed entirely in the cloud (cloud environment), or entirely on the edge (edge ​​environment), or on the terminal. In other examples, the task processing system 10 can also be deployed in different environments, for example, it can be deployed in the cloud and the edge in an edge-cloud collaborative manner. The following is an example of the deployment of the task processing system 10 in an edge-cloud collaborative manner.

[0110] See also Figure 2 A schematic diagram of a deployment form of a task processing system 10 is shown, as shown in Figure 2 As shown, the deployment of the task processing system 10 may involve hardware devices in the cloud and hardware devices at the edge. The hardware devices in the cloud usually have rich computing power and huge storage capacity, such as servers and databases, and the hardware devices at the edge have certain reasoning and storage capabilities, such as industrial computers and artificial intelligence edge computing devices. In some possible implementations, the deployment of the task processing system 10 also involves terminals, which may include mobile phones, self-driving cars, industrial quality inspection systems and other IoT devices.

[0111] exist Figure 2 In the example, a pluggable unknown task training module 102 that supports model deployment is deployed in the cloud. The input of the unknown task training module 102 can be the initial task model of the edge working node, known task samples and unknown task samples stored in the cloud. The function of the unknown task training module 102 is to identify the input initial task model, and adapt the corresponding weight update strategy according to the recognition result (such as the task type determined by the reasoning result of at least one task sample based on the initial task model). Furthermore, the corresponding performance evaluation index and model reliability evaluation index can also be adapted according to the recognition result, and then the initial task model is trained based on the instance weighting method, and the weight of the trained task model is output, and the task model reliability is additionally output; the retrained task model can adapt to the unknown task data distribution and perform unknown task processing.

[0112] Similarly, the unknown task strategy search module 104 that supports pluggable models can also be deployed in the cloud. The input of the unknown task strategy search module 104 can be the initial task model of the edge working node and the unknown task sample stored in the cloud. The function of the unknown task strategy search module 104 is to identify the initial task model, call the adapted model evaluation index and post-processing method, search for sample processing strategies that are helpful for model reasoning, and output the preprocessing strategy. Furthermore, the unknown task strategy search module 104 can also additionally output the reliability of the preprocessing strategy. The preprocessing strategy searched by the unknown task strategy search module 104 can preprocess unknown samples when performing task reasoning at the edge, thereby improving the processing performance of the task model for unknown samples.

[0113] The unknown task processing module 103 and the unknown task preprocessing module 105 can be deployed at the edge. Among them, the input of the unknown task preprocessing module 105 can be the unknown task reasoning data (i.e., unknown samples to be processed) provided by the terminal (e.g., the application running on the terminal) and the preprocessing strategy output by the cloud. The function of the unknown task preprocessing module 105 is to call the preprocessing strategy to preprocess the unknown tasks to be processed, and improve the processing performance of the subsequent task model for unknown tasks. The input of the unknown task processing module 103 can be the unknown task samples enhanced by the unknown task preprocessing module 105 and the task model output by the cloud. The function of the unknown task processing module 103 is to call the task model to process the enhanced unknown task samples, for example, calling the task model to reason and fuse the enhanced unknown task samples, and return the unknown task reasoning results to the terminal (the application running on the terminal).

[0114] Interaction Module( Figure 1 , Figure 2The interactive module deployed on the cloud side is used to receive known task samples and unknown task samples, and can also receive the initial task model, and the interactive module deployed on the edge side is used to receive unknown tasks to be processed, and further, can also display the unknown task reasoning results.

[0115] It should be noted that Figure 2 This is only an example of the deployment form of the task processing system 10. In other possible implementations of the present application, the task processing system 10 may also adopt other deployment methods, and the present application does not limit this.

[0116] based on Figure 1 The task processing system 10 shown in the figure, the present application also provides a task processing method, and the task processing method of the present application is introduced from the perspective of the task processing system 10 below.

[0117] See also Figure 3 The flowchart of the task processing method shown in FIG. 1 can be executed by the task processing system 10. The task processing system 10 is used to provide a task model for processing unknown tasks, and to process the unknown tasks based on the task model. The method includes:

[0118] S302 : The task processing system 10 receives an initial task model, known task samples, and unknown task samples input by a user.

[0119] The initial task model can be the user's own business model. The unknown task training module 102 in the task processing system 10 can adapt the user's own business model according to the needs, that is, the unknown task training module 102 is decoupled from the user's own business model. Therefore, the task processing system 10 can receive the initial task model input by the user, so as to perform training based on the initial task model and obtain a task model for processing unknown tasks.

[0120] The known task samples may be samples within the distribution, and the unknown task samples may be samples outside the distribution. The known task samples and the unknown task samples carry annotation information. Depending on the type of the initial task model or the task, the annotation information carried by the task samples (such as known task samples and unknown task samples) may be different. For example, when the initial task model is a classification model, the annotation information carried by the task samples may include category labels. For another example, when the initial task model is a target detection model, the annotation information carried by the task samples may include a label box and a category label. Among them, the label box is used to indicate the location of the target.

[0121] S304 , the task processing system 10 inputs at least one task sample into the initial task model for reasoning, and determines a weight update strategy according to the reasoning result.

[0122] The task processing system 10 can extract at least one task sample (known task sample or unknown task sample), and use the task model input by the user to perform reasoning to obtain the reasoning result of the initial task model on the task sample, and then the task processing system 10 can perform task identification according to the reasoning result, for example, the format of the reasoning result, to obtain the task type. The task processing system 10 can update the strategy according to the weight corresponding to the determined task type.

[0123] The task type is used to identify tasks of different categories. Specifically, the task type may include any of the following: a classification task, a detection task (for example, a target detection task), or a segmentation task. When the inference result includes a category, or the inference result includes a category and a scalar with a value of 0-1 (characterizing the reliability of the category), the task type may be a classification task. When the inference result includes a category and a position coordinate, or the inference result includes a category, a position coordinate, and two scalars with values ​​of 0-1, the task type may be a detection task. When the inference result includes a matrix of the same size as the input image, and the elements in the matrix include categories, or include categories and a scalar with values ​​of 0-1, the task type may be a segmentation task.

[0124] The weight update strategy corresponding to the classification task includes a sample-level update strategy, the weight update strategy corresponding to the detection task includes an instance-level update strategy, and the weight update strategy corresponding to the segmentation task includes a pixel-level update strategy. Based on this, the task processing system 10 can determine the weight update strategy corresponding to the task type according to the mapping relationship between the task type and the weight update strategy. For example, when the task processing system 10 determines that the task type is a classification task, it can determine that the weight update strategy corresponding to the task type is a sample-level update strategy. Among them, the sample-level update strategy, the instance-level update strategy, and the pixel-level update strategy are update strategies of different granularities, and an instance can refer to a target detected in a sample.

[0125] S306 , the task processing system 10 inputs known task samples and unknown task samples into the initial task model, performs new model training through a weight update strategy, and obtains a task model.

[0126] Specifically, the task processing system 10 can input known task samples and unknown task samples into the initial task model, iteratively update the weights according to the inference results of the initial task model on the task samples and the labeling information of the samples, and when the model after the weight update meets the set conditions, the model after the weight update can be output as the task model to process unknown tasks. Among them, when updating the weights, the task processing system 10 can adopt a weight update strategy corresponding to the task type. For example, when the task type is a classification task, the task processing system 10 can adopt a sample-level weight update strategy to update the weights of the initial task model. Among them, the task processing system 10 adopts a sample-level weight update strategy to update the weights, specifically calculating the loss value based on the inference results of the task samples, for example, calculating the loss value based on the inference results and labeling information of a batch of task samples (one or more task samples), and then updating the weights according to the loss value.

[0127] In some possible implementations, the performance evaluation index corresponding to the classification task includes the F value (e.g., F1-score), the performance evaluation index corresponding to the detection task includes the mean average precision (mAP), and the performance evaluation index corresponding to the segmentation task includes the mean intersection over union (mIOU). mIOU is generally calculated based on the class, and by taking the average value of the IOU of each class, a global performance evaluation can be achieved. The task processing system 10 can also determine the performance evaluation index based on the reasoning result. For example, the task processing system 10 can determine the task type based on the reasoning result, and then determine the performance evaluation index corresponding to the task type. Accordingly, the task processing system 10 can filter out the task model from the transient model obtained by training according to the performance evaluation index. For example, the task processing system 10 can divide the unknown task sample and the known task sample to obtain a training set and a validation set. The training set includes the first sample in the unknown task sample and the known task sample, and the validation set includes the second sample in the unknown task sample. The task processing system can iteratively update the weight of the initial task model through the weight update strategy according to the samples in the training set to obtain a transient model. The task processing system 10 can verify the transient model (such as the model that meets the training stop condition) on the validation set. Specifically, the task processing system 10 can evaluate the performance of the transient model through the performance evaluation index based on the samples in the validation set to obtain the performance value. When the performance value of the transient model reaches the performance threshold or the number of iterations of the initial task model reaches the maximum number of iterations, the transient model can be output as the task model. When the performance value does not reach the performance threshold or the number of iterations of the initial task model does not reach the maximum number of iterations, the model training can continue.

[0128] In some possible implementations, the model reliability evaluation index corresponding to the classification task includes the classification error rate (for example, the sample classification error), and the model reliability evaluation index corresponding to the detection task includes the regression error rate (for example, the instance regression error rate) and the classification error rate (for example, the instance classification error rate). The model reliability evaluation index corresponding to the segmentation task includes the classification error rate (for example, the pixel classification error rate). Accordingly, the task processing system 10 can also determine the model reliability evaluation index based on the reasoning result, for example, the task type can be determined based on the reasoning result, and then the model reliability evaluation index corresponding to the task type can be determined. The task processing system 10 determines the reliability of the task model based on the samples in the test set through the model reliability evaluation index. Further, the task processing system 10 can output the reliability of the task model.

[0129] S308 , the task processing system 10 obtains a preprocessing strategy according to the unknown task sample, the initial task model and the task type determined by the reasoning result.

[0130] The preprocessing strategy refers to a strategy for enhancing unknown tasks in the inference phase. The preprocessing strategy may include at least one sample processing strategy. The sample processing strategy includes a sample processing function and the strength of the sample processing function, which can be expressed in the following format: (function name N, strength S). ​​The function name N can also be a method name, and the strength can be the degree of deformation of the current sample processing strategy, ranging from 0 to 1. The sample processing strategy ('Flip', 0.5) means that a sample has a 50% probability of being horizontally flipped, and the sample processing strategy ('Rotate', 0.3) is a clockwise rotation of 108 degrees (0.6*360 degrees).

[0131] Specifically, the unknown task samples can be divided into a training set and a validation set. For example, the first sample in the unknown task samples is divided into a training set, and the second sample in the unknown task samples is divided into a validation set. The task processing system 10 can process the original samples in the validation set through a sample processing strategy to obtain enhanced samples. The original samples include the second sample in the unknown task samples, and the enhanced samples are samples processed using the sample processing strategy. The task processing system 10 can obtain a preprocessing strategy based on the inference results of the original samples and the enhanced samples by the initial task model. The preprocessing strategy includes a sample processing strategy in which the second performance value of the enhanced sample reasoning is greater than the first performance value of the original sample reasoning, wherein the performance value can be an indicator value of a performance evaluation indicator determined according to the task type.

[0132] In some possible implementations, the task processing system 10 may determine a search space, the search space including at least one sample processing strategy, and then sample the sample processing strategy from the search space, and use the sampled sample processing strategy to process the samples in the validation set to obtain an enhanced validation set. Then the task processing system 10 obtains a first performance value (for example, for a detection task, the first performance value may be a first accuracy, or referred to as a first precision) based on the first reasoning result of the task model for the samples in the validation set, and obtains a second performance value based on the second reasoning result of the task model for the samples in the enhanced validation set. The task processing system 10 may determine a preprocessing strategy based on the first performance value and the second performance value. The preprocessing strategy may include a sample processing strategy in which the second performance value is greater than the first performance value.

[0133] Further, the task processing system 10 can also determine the post-processing method corresponding to the task type. The post-processing method may include but is not limited to maximum confidence and non-maximum suppression. Accordingly, the task processing system 10 can fuse the second reasoning result (the reasoning result of the enhanced sample) and the first reasoning result (the reasoning result of the original sample) according to the post-processing method corresponding to the task type, and then obtain the second performance value according to the fused reasoning result. In some possible implementations, the task processing system 10 can also determine the reliability of the pre-processing strategy and output the reliability of the pre-processing strategy. Specifically, the task processing system 10 can process the samples in the test set according to the sample processing strategy to obtain the processed test samples, and determine the strategy reliability according to the reasoning results of the processed test samples and the labeling information of the samples in the test set based on the initial task model. For example, the task processing system 10 can process the samples in the test set according to the sample processing strategy in which the second performance value is greater than the first performance value, and obtain the processed test samples, and then the task processing system 10 infers the processed test samples according to the task model to obtain the third reasoning result, and determine the strategy reliability according to the third reasoning result and the labeling information of the samples in the test set.

[0134] Taking into account the situation where the system's built-in sample processing strategy is difficult to meet the needs, the task processing system 10 can also receive user-defined sample processing strategies. Then the task processing system 10 can add the user-defined sample processing strategies to the search space and perform strategy search, thereby flexibly expanding the unknown task processing capabilities.

[0135] It should be noted that S308 is only an optional step of the embodiment of the present application. The unknown task processing method of the present application can obtain the preprocessing strategy in other ways. For example, the task processing system can obtain the preprocessing strategy based on the initial task model and the unknown task sample search, without reusing the inference results or task types output during the unknown task training. Alternatively, the unknown task processing method of the present application can be executed without obtaining the preprocessing strategy.

[0136] S310 , the task processing system 10 uses a preprocessing strategy to enhance the unknown task to be processed.

[0137] Specifically, the task processing system 10 can process the unknown task to be processed according to the sample processing function and strength in the preprocessing strategy, thereby enhancing the unknown task to be processed.

[0138] It should be noted that the task processing method of the embodiment of the present application may not perform S308 to S310. For example, when processing unknown tasks to be processed, the task processing system 10 may input the unknown tasks to be processed into the task model for processing without performing preprocessing such as enhancement on the unknown tasks.

[0139] S312 , the task processing system 10 processes the enhanced unknown task through the task model.

[0140] Specifically, the task processing system 10 may input the enhanced unknown task into the task model, and the task model may infer the enhanced unknown task, thereby implementing unknown task processing. The task processing system 10 may output the unknown task inference result.

[0141] It should be noted that when the task processing system 10 does not execute S308 to S310, the task processing system 10 can directly input the unknown task to be processed into the task model, and the task model infers the unknown task to be processed, thereby realizing unknown task processing. In addition, the task processing method of the present application may not execute S312. For example, if no unknown task is received within a period of time, the task processing system 10 may not execute S312.

[0142] Based on the above description, it can be known that the embodiment of the present application provides a model-independent (model pluggable) unknown task processing method, which adapts the weight update strategy by identifying the output of the initial task model (for example, the reasoning result of the task sample), so as to achieve unknown task training based on instance re-weighting with any type of initial task model as input, and has the characteristic of model pluggability. Furthermore, the method can also adapt the model reliability evaluation index, and accordingly, the reliability of the task model can also be output based on the model reliability evaluation index, so as to provide feedback to the user on the status of unknown task processing. When the reliability of the task model is low, the user can be fed back to collect and annotate more unknown task samples to train a task model that is effective enough for the current unknown task processing. In addition, the embodiment of the present application also introduces an enhancement method in the task model reasoning stage to improve the detection performance of the task model for unknown tasks in real time.

[0143] Next, the model training and strategy search processes are described in detail from the perspectives of the unknown task training module 102 and the unknown task strategy search module 104 in the task processing system 10 .

[0144] First, see Figure 4 The flowchart of an unknown task training module 102 for training a task model shown includes the following steps:

[0145] (1) The unknown task training module 102 obtains known task samples (known data) and unknown task samples (unknown data) input by the user, divides the unknown task samples into a training set, a validation set and a test set, and merges the training set including the unknown task samples with the known task samples into a new training set. Then, by calculating the proportion of the unknown task samples in the training set, the unknown task samples are assigned initialization weights.

[0146] For ease of understanding, the weight initialization process is explained below with examples.

[0147] For a classification task, assuming that among the training data received by the unknown task training module 102, 1,000 known task samples and 20 unknown task samples are divided into a training set, and the ratio of unknown task samples to known task samples in the training set is 50:1. Accordingly, the unknown task training module 102 can assign a 50-fold weight to the unknown task samples.

[0148] For a target detection task, assuming that in the training data received by the unknown task training module 102, 1000 known label boxes and 50 unknown label boxes are divided into a training set, and the ratio of known label boxes to unknown label boxes in the training set is 20:1. Accordingly, the unknown task training module 102 can assign a 20-fold weight to the unknown task samples.

[0149] (2) The unknown task training module 102 extracts a single piece of reasoning data (e.g., a single task sample) from the training set, and uses the initial task model input by the user for reasoning, performs task identification on the output result of the initial task model (e.g., the reasoning result), and obtains the task type.

[0150] (3) The unknown task training module 102 automatically adapts the weight update strategy, performance evaluation index and model reliability evaluation index corresponding to the task type according to the recognition result (recognized task type) of step (2).

[0151] (4) The unknown task training module 102 uses the training set and weights generated in step (1) to train the initial task model and temporarily stores the weights of the trained task model.

[0152] The weight of the task model after training can be the weight of the initial task model updated by the weight update strategy according to the samples in the training set. Specifically, the unknown task training module 102 can input the samples in the training set into the initial task model, and update the weight of the initial task model by the weight update strategy. When the updated model meets the training stop condition, the iterative training can be stopped to obtain the weight of the task model after training. The model after the weight update is also called a transient model.

[0153] (5) The unknown task training module 102 uses the performance evaluation index adapted in step (3) to evaluate the performance of the trained model (transient model) according to the validation set generated in step (1). When the performance of the trained model reaches the performance threshold or the number of iterations of the initial task model reaches the maximum number of iterations, the process proceeds to step (7); otherwise, the process proceeds to step (6) to continue iterative training.

[0154] (6) The unknown task training module 102 determines the new round of weights of the model on the training set based on the weight update strategy adapted in step (3), trains the model, temporarily stores the weights of the task model after training, and returns to step (5).

[0155] (7) The unknown task training module 102 uses the model reliability evaluation index adapted in step (3) to determine the reliability of the task model on the test set divided in step (1), and outputs the task model weight and the reliability of the task model.

[0156] Figure 3 The difference between the traditional unknown task training method and the unknown task training method of the present application is also shown, as shown below:

[0157] The basic idea of ​​the traditional unknown task training method is to use weak classifiers to classify samples and continuously improve the accuracy of the classifier through iterative weighting. In each iteration, the classifier results and error rate are used as input, the weight of each classifier is calculated, and the sample weight is updated according to the weight. Finally, the results of all classifiers are weighted averaged to obtain the final classification result.

[0158] Compared with the traditional unknown task training method, the unknown task training module 102 of the embodiment of the present application can realize unknown task training based on instance re-weighting by using any task model as input by identifying the task model output, adapting the corresponding weight update strategy, performance evaluation index and model reliability evaluation index, and has the feature of pluggable model. In addition, the unknown task training module 102 additionally outputs the reliability of the task model and provides feedback to the user on the unknown task processing status. When the reliability of the task model is low, the user is fed back that they should collect and annotate more unknown task samples in order to train a task model that is effective enough for the current unknown task processing. In this way, the module solves the problem that the traditional unknown task training method cannot achieve pluggable models and unknown task processing status feedback, and has high business value.

[0159] Secondly, see Figure 5 The flowchart of the unknown task strategy search module 104 shown includes the following steps:

[0160] (1) The unknown task strategy search module 104 obtains unknown task samples input by the user and divides the unknown task samples into a verification set and a test set.

[0161] It should be noted that the unknown task strategy search module 104 can also be reused Figure 4 Validation set and test set in the illustrated embodiment.

[0162] (2) The unknown task strategy search module 104 extracts a single piece of reasoning data (eg, a single task sample) from the validation set, uses the initial task model input by the user for reasoning, performs task identification on the output result of the initial task model, and obtains the task type.

[0163] Similarly, the unknown task strategy search module 104 can also be reused Figure 4 The identification result of the initial task model in the illustrated embodiment (eg, the identified task type) is shown.

[0164] (3) The unknown task strategy search module 104 adapts the performance evaluation index, post-processing method, and strategy reliability evaluation index corresponding to the task type according to the recognition result of step (2).

[0165] Among them, the performance evaluation indicators corresponding to the task type can also be reused Figure 4 The performance evaluation index determined by the unknown task training module 102 in the illustrated embodiment.

[0166] (4) When the user inputs a customized sample processing strategy, the unknown task strategy search module 104 merges the sample processing strategy input by the user with the default sample processing strategy to form a search space.

[0167] (5) The unknown task strategy search module 104 calculates the basic accuracy (eg, the first accuracy) of the input initial task model based on the validation set provided in step (1) and the performance evaluation index adapted in step (3).

[0168] (6) The unknown task strategy search module 104 samples a sample processing strategy from the search space, and then uses the sampled sample processing strategy to perform data enhancement on the validation set, and then calls the input initial task model to perform reasoning on the enhanced validation set and the original validation set to obtain two sets of reasoning results. The reasoning results of the enhanced validation set and the original validation set are fused based on the post-processing method adapted in step (3), and the accuracy of the fused result (e.g., the second accuracy) is calculated.

[0169] The sample processing strategy may include a sample processing function and strength. The unknown task strategy search module 104 may sample a sample processing strategy from the search space by random sampling or sequential sampling.

[0170] (7) If the new accuracy is less than or equal to the basic accuracy, it is considered that the sampled sample processing strategy (sample processing function or strength) is not suitable for the unknown task, and the unknown task strategy search module 104 re-samples the strategy; if the new accuracy is greater than the basic accuracy, the unknown task strategy search module 104 subtracts the new accuracy from the basic accuracy to obtain the contribution of the sampled sample processing strategy, and stores the sampled sample processing strategy (sample processing function and strength) and contribution.

[0171] (8) The unknown task strategy search module 104 determines whether the maximum number of iterations has been reached. If not, return to step (6). If so, the unknown task strategy search module 104 first sorts the stored sample processing strategies by contribution, and retains strategies whose contribution satisfies the conditions (for example, greater than the preset contribution, or ranked as top n, where n is a positive integer). The unknown task strategy search module 104 uses the retained sample processing strategies to enhance the test set in step (1), calls the input initial task model to reason on the enhanced test set and the original test set, fuses the reasoning results based on the post-processing method adapted in step (3), calculates the reliability of the preprocessing strategy, and outputs the preprocessing strategy of the unknown task and the reliability of the preprocessing strategy.

[0172] The preprocessing strategy for the unknown task may include a sample processing strategy whose contribution satisfies the above conditions.

[0173] (9) If the reliability of the pre-processing strategy is low, the unknown task strategy search module 104 can expand the unknown task processing capability according to the sample processing strategy updated by the user.

[0174] Figure 5 The difference between the traditional reasoning stage enhancement method and the reasoning stage enhancement method in the unknown task strategy search module 104 of the present application is also shown, as shown below:

[0175] The traditional enhancement method in the inference stage is to perform data enhancement operations such as horizontal flipping, contrast enhancement, and rotation angle on the original image during the model inference stage, obtain multiple images for inference separately, and then fuse and post-process the multiple results to obtain the final output result.

[0176] Compared with the traditional reasoning stage enhancement method, the unknown task strategy search module 104 proposed in the embodiment of the present application can realize the enhanced strategy search in the reasoning stage with any task model as input by identifying the task model output, adapting the corresponding post-processing method, performance evaluation index, and strategy reliability evaluation index, and has the characteristics of pluggable model. In addition, the unknown task strategy search module 104 has an additional optional input user's sample processing strategy, and the reliability of the additional output pre-processing strategy. The user's sample processing strategy input increases the flexibility of the unknown task strategy search module 104. The user can use the built-in sample processing strategy of the unknown task strategy search module 104 to search for unknown task pre-processing strategies, or input a custom sample processing strategy to search together with the built-in sample processing strategy (such as the default sample processing strategy) to expand the unknown task processing capability.

[0177] The unknown task strategy search module 104 additionally outputs the reliability of the preprocessing strategy, which is used to indicate the confidence of the searched preprocessing strategy for unknown task processing. When the reliability is low, the user is fed back to input more sample processing strategies to search for unknown task preprocessing strategies that are sufficiently effective for unknown tasks. In this way, the module solves the problems that the traditional reasoning stage enhancement method cannot achieve model pluggability, unknown task processing status feedback, and expansion of unknown task processing capabilities, and has high business value.

[0178] It should be noted that Figure 4 The method steps of the embodiment shown are Figure 5 The method steps of the illustrated embodiment can be executed in parallel, or in sequence according to a set order. Figure 4 The embodiment shown or Figure 5 Part of the data or intermediate results in the illustrated embodiment may be reused.

[0179] Next, the task processing method of this application is introduced in combination with specific application scenarios.

[0180] See also Figure 6 The schematic diagram of a scenario in which a task processing method is applied to a visual inspection task for surface defects of industrial products is shown. The visual inspection task for surface defects of industrial products may include the following stages:

[0181] Phase 1: Data collection: In this embodiment, an industrial camera collects red, green, blue (RGB) images into a production database.

[0182] Phase 2: Data labeling: Defects are labeled on the RGB images of the surfaces of industrial products in the production database by a manual or automatic labeling system, and the labeled images are stored in the training database.

[0183] Phase 3: Task model training and launch: Use the labeled data in the training database (such as labeled images) to train the task model for quality inspection and deploy the task model into the quality inspection process.

[0184] Stage 4: Unknown tasks caused by changes in raw materials, production processes, quality inspection environment and other factors cause the detection performance of the task model to decline, and enter the unknown task processing process.

[0185] The unknown task processing process may include an unknown task training process and an unknown task preprocessing strategy search process.

[0186] The unknown task preprocessing strategy search process includes the following steps:

[0187] (1) The unknown task strategy search module 104 receives the initial task model for industrial quality inspection and unknown task samples input by the user. The unknown task samples can be divided into a verification set and a test set.

[0188] (2) The unknown task strategy search module 104 performs task identification. Specifically, the unknown task strategy search module 104 can call the model reasoning interface of the initial task model to infer the unknown task sample and determine the task type based on the output result. In some examples, if the output is a one-dimensional vector or scalar, the task type is identified as a classification task; if the output is a two-dimensional array or a nested list, the task type is identified as a target detection task; if the output is a higher-dimensional array, the task type is identified as a segmentation task. In this embodiment, the unknown task strategy search module 104 identifies the current task type as a target detection task.

[0189] (3) The unknown task strategy search module 104 performs task adaptation. Specifically, the unknown task strategy search module 104 adapts the performance evaluation index, post-processing method, and strategy reliability evaluation index according to the recognition result. For example, if the recognition result is a classification task, the unknown task strategy search module 104 determines that the performance evaluation index is F1-score, the post-processing method is maximum confidence, and the strategy reliability evaluation index is the classification error rate. In this embodiment, the recognition result is a target detection task, and accordingly, the performance evaluation index is MeanAverage Precision, the post-processing method is non-maximum suppression, and the strategy reliability evaluation indicators are regression error rate and classification error rate.

[0190] (4) The unknown task strategy search module 104 determines the search space. If the user inputs a customized sample processing strategy, it is merged into the default sample processing strategy space to form a search space. In this embodiment, the default sample processing strategy space includes 19 methods, namely: 'Color', 'Contrast', 'Brightness', 'Sharpness', 'AutoContrast', 'Equalize', 'Resize', 'Flip', 'Grey', 'Invert', 'Posterize', 'Solarize', 'HSV', 'Rotate', 'ShearX', 'ShearY', 'PepperNoise', 'GaussNoise', 'Cutout'.

[0191] (5) The unknown task strategy search module 104 initializes the enhanced strategy list L, and obtains the basic accuracy P0 using the initial task model reasoning validation set based on the adapted MeanAveragePrecision evaluation indicator.

[0192] (6) The unknown task strategy search module 104 randomly samples a sample processing strategy (also called data enhancement strategy M) from the search space in the format of (method name N, strength S). ​​The strength S is the degree of deformation of the current data enhancement strategy, ranging from 0 to 1. For example, ('Flip', 0.5) means that a sample has a 50% probability of being horizontally flipped; ('Rotate', 0.3) means rotating 108 degrees clockwise.

[0193] (7) The unknown task strategy search module 104 uses the initial task model to perform reasoning on the M-enhanced validation set and the original validation set, respectively, and fuses the reasoning results based on the adapted non-maximum suppression post-processing method. Finally, the accuracy P1 of the fused result is evaluated based on the adapted MeanAverage Precision indicator. If P1 is greater than P0, the contribution C1=P1-P0 is recorded, and the data enhancement strategy M is updated to (method name N, strength S, contribution C), and stored in the enhancement strategy list L. If P1 is less than P0, it is determined whether the termination condition of the maximum number of iterations is reached. If not, it returns to step (6) to continue iterating.

[0194] (8) The unknown task strategy search module 104 arranges the elements in the enhancement strategy list L in descending order according to the contribution C, retains the first K entries in the enhancement strategy list, and the initial task model uses the retained K strategies to calculate the strategy reliability on the test set. In this embodiment, the strategy reliability evaluation index is the regression error rate + classification error rate. The calculation process is as follows: 1) All images in the test set are looped. For a certain image I in the test set, the unknown task strategy search module 104 calls the strategy in the enhancement strategy list L to enhance I and generates Len(L) pieces of enhancement data, where Len(L) is the list length. 2) The initial task model infers Len(L) pieces of enhancement data and the original image I, and uses the non-maximum suppression post-processing method to fuse the inference results. 3) Calculate the error rate of the label box in image I. Take the label box B belonging to category 1 as an example, obtain the prediction result R with the largest intersection-over-union ratio with B in the inference result, assuming that the score of R belonging to category 1 is RS, and the intersection-over-union ratio of R and B is RI, then the classification reliability of the label box B is ConfidenceC=RS, and the regression reliability is ConfidencR=RI. 4) Determine the average classification reliability and the average regression reliability of all label boxes.

[0195] (9) Output the search-completed enhancement strategy list L (the preprocessing strategy includes the sample processing strategy in the enhancement strategy list L), as well as the classification reliability and regression reliability of the output preprocessing strategy.

[0196] The unknown task training process includes the following steps:

[0197] (1) The unknown task training module 102 receives the initial task model, known task samples and unknown task samples for industrial quality inspection input by the user, divides the unknown task samples into a training set, a validation set and a test set, and merges the training set including the unknown task samples with the known task samples into a new training set.

[0198] (2) The unknown task training module 102 performs task identification and task adaptation. In this embodiment, the unknown task training module 102 can inherit the recognition result of the unknown task strategy search module 104, and there is no need to perform repeated recognition. The unknown task training module 102 adapts the weight update strategy, performance evaluation index, and model reliability evaluation index related to the task. For example, if the recognition result is a classification task, the weight update strategy can be a sample-level weight update strategy, the performance evaluation index is F1-score, and the model reliability evaluation index is the classification error rate. In this embodiment, the recognition result is a target detection task, the weight update strategy is an instance-level weight update strategy, the performance evaluation index is mean Average Precision, and the model reliability evaluation index is the regression error rate and the classification error rate. It should be noted that the unknown task training module 102 can inherit the adaptation result of the unknown task strategy search module 104, for example, inherit the performance evaluation index adapted by the unknown task strategy search module 104.

[0199] (3) The unknown task training module 102 calculates the proportion of unknown task samples in the training set, assigns initialization weights to the unknown task samples, and temporarily stores the data weights.

[0200] (4) The unknown task training module 102 uses the temporarily stored data weights to train the input initial task model and temporarily stores the updated model weights.

[0201] (5) The unknown task training module 102 uses the adapted mAP indicator to evaluate the performance R of the updated model weights on the unknown task validation set.

[0202] (6) The unknown task training module 102 determines whether the maximum number of iterations has been reached or whether R is greater than the performance threshold. If not, an adapted weight update strategy is used to calculate the data weight for the next training of the task model. In this embodiment, the weight update strategy is an instance-level weight update strategy. Taking the label box B belonging to category 1 as an example, the prediction result P with the largest intersection-over-combination ratio with B in the task model prediction result is obtained. Assuming that the score of P belonging to category 1 is PS, and the intersection-over-combination ratio of P and B is PI, then the error rate of label box B ErrorB=((1-PS)+(1-PI)) / 2. If B is the label of unknown data, then the weight of label box B for the next training is WeightB=1+ErrorB; if B is the label of known data, then the weight of label box B for the next training is WeightB=1-ErrorB. Calculate all training data label boxes in a loop to generate the data weight for the next task model training, temporarily store the data weight, and return to step (4).

[0203] (7) The unknown task training module 102 uses the adapted model reliability evaluation index to evaluate the reliability of the current model on the test set. This embodiment is a target detection task, and the strategy reliability evaluation index is regression reliability (characterized by regression error rate) and classification reliability (characterized by classification error rate). The calculation process is: loop all images in the test set, for a certain image I in the test set, take the label box B belonging to category 1 as an example, obtain the prediction result R with the largest intersection and union ratio with B in the inference result, assuming that the score of R belonging to category 1 is RS, and the intersection and union ratio of R and B is RI, then the classification reliability of the label box B is ConfidenceC=RS, and the regression reliability is ConfidenceR=RI. Average the classification reliability and regression reliability of all label boxes.

[0204] (8) The unknown task training module 102 outputs the weight of the trained task model, as well as the classification reliability and regression reliability of the output task model.

[0205] See also Figure 7 Another task processing method shown is applied to an autonomous driving task or an industrial product surface defect visual inspection task. In this scenario, the behavior of the internal modules of the task processing system 10 is similar to Figure 6 The illustrated embodiments are similar, so the entire unknown task processing platform is represented here.

[0206] Different from Figure 6 The pure cloud architecture of the embodiment shown, Figure 7 The illustrated embodiment adopts an edge-cloud collaborative architecture. Specifically, the unknown task processing platform is located in the cloud, while the user is located at the edge. In this embodiment, the pluggable task processing system 10 receives the business data uploaded by the user with the help of the shared storage service in the cloud, and at the same time annotates the business data uploaded by the user in the cloud based on the annotation system, so as to prepare for unknown task training for the initial task model uploaded by the user. The user does not need to know the internal details of the unknown task processing platform, but only needs to upload the initial task model to the unknown task processing platform, and the unknown task processing platform can return the task model that can adapt to the unknown task and the preprocessing strategy that is helpful for unknown task reasoning.

[0207] also, Figure 7 The processing object of the embodiment shown may be different from Figure 6 In this embodiment, the pluggable unknown task processing platform provides services in the cloud, and the number of users is unlimited, for example, it can be a multi-party collaboration scenario, and Figure 6 The embodiments shown are generally for a unilateral development scenario. Figure 7As shown in the figure, the pluggable unknown task processing platform allows users to input the initial task model. Therefore, no matter whether the user inputs the U-Net model based on autonomous driving segmentation or the YOLOX model based on industrial quality inspection, it can be processed after uploading to the pluggable unknown task processing platform, which solves the problem that the unknown task training technology based on model coupling and the inference stage enhancement technology are difficult to achieve model pluggability.

[0208] Based on the above-mentioned task processing method, the present application also provides a task processing system 10. The task processing system 10 is used to provide a task model for processing unknown tasks, and to process unknown tasks based on the task model. Figure 1 As shown, the task processing system 10 includes:

[0209] Interaction Module( Figure 1 (not shown), for receiving an initial task model, known task samples, and unknown task samples input by a user;

[0210] The unknown task training module 102 is used to input at least one task sample into the initial task model for reasoning, determine the weight update strategy according to the reasoning result, input the known task sample and the unknown task sample into the initial task model, perform model training through the weight update strategy, and obtain the task model.

[0211] Among them, exemplarily, the above-mentioned interaction module and unknown task training module 102 can be implemented by hardware, or can be implemented by software.

[0212] When implemented by software, the interaction module and the unknown task training module 102 may be applications running on a computing device (such as a server). For example, the unknown task training module 102 may be a computing engine running on a computing device, etc. The application may be provided to users in the form of a virtualized service. Virtualization services may include virtual machine (VM) services, bare metal server (BMS) services, and container services. Among them, the VM service may be a service that uses virtualization technology to virtualize a virtual machine (VM) resource pool on multiple physical hosts (such as computing devices) to provide users with VMs on demand for use. The BMS service is a service that virtualizes a BMS resource pool on multiple physical hosts to provide users with BMS on demand for use. The container service is a service that virtualizes a container resource pool on multiple physical hosts to provide users with containers on demand for use. VM is a simulated virtual computer, that is, a logical computer. BMS is a high-performance computing service that can be elastically scalable, and its computing performance is no different from that of a traditional physical machine, and it has the characteristics of secure physical isolation. Container is a kernel virtualization technology that can provide lightweight virtualization to achieve the purpose of isolating user space, processes and resources. It should be understood that the VM service, BMS service and container service in the above virtualization services are only used as specific examples. In actual applications, the virtualization service can also be other lightweight or heavyweight virtualization services, which are not specifically limited here.

[0213] When implemented by hardware, the interaction module can be implemented by a transceiver or other transceiver module. The unknown task training module 102 may include at least one computing device, such as a server, etc. Alternatively, the unknown task training module 102 may also be a device implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0214] In some possible implementations, the known task samples are samples that are successfully inferred by the initial task model, and the unknown task samples are samples that are failed to be inferred by the initial task model.

[0215] In some possible implementations, the unknown task training module 102 is further used to:

[0216] Determine a performance evaluation indicator according to the reasoning result;

[0217] The unknown task training module 102 is specifically used for:

[0218] According to the samples in the training set, iteratively updating the weight of the initial task model through the weight update strategy to obtain a transient model, wherein the training set includes the first sample in the unknown task sample and the known task sample;

[0219] According to the samples in the validation set, the transient model is subjected to performance evaluation by the performance evaluation index to obtain a performance value, wherein the validation set includes the second sample in the unknown task sample;

[0220] When the performance value of the transient model reaches a performance threshold or the number of iterations of the initial task model reaches a maximum number, the transient model is output as the task model.

[0221] In some possible implementations, the unknown task training module 102 is further used to:

[0222] Determine a model reliability evaluation index according to the inference result;

[0223] According to the samples in the test set, the reliability of the task model is determined by using the model reliability evaluation index.

[0224] In some possible implementations, the unknown task training module 102 is specifically used to:

[0225] Determine the task type according to the inference result, and determine the weight update strategy corresponding to the task type;

[0226] The task type includes a classification task, a detection task or a segmentation task. The weight update strategy corresponding to the classification task includes a sample-level update strategy, the weight update strategy corresponding to the detection task includes an instance-level update strategy, and the weight update strategy corresponding to the segmentation task includes a pixel-level update strategy.

[0227] In some possible implementations, the performance evaluation indicators corresponding to the classification task include the F value, the model reliability evaluation indicators corresponding to the classification task include the sample classification error rate, the performance evaluation indicators corresponding to the detection task include the mean average precision mAP, the model reliability evaluation indicators corresponding to the detection task include the instance regression error rate and the instance classification error rate, the performance evaluation indicators corresponding to the segmentation task include the average intersection-over-union ratio mIOU, and the model reliability evaluation indicators corresponding to the segmentation task include the pixel classification error rate.

[0228] In some possible implementations, the system 10 further includes:

[0229] An unknown task strategy search module 104 is used to obtain a preprocessing strategy according to the unknown task sample, the initial task model and the task type determined by the reasoning result;

[0230] An unknown task preprocessing module 105, used to enhance the unknown task to be processed using the preprocessing strategy;

[0231] The unknown task processing module 103 is used to process the enhanced unknown task through the task model.

[0232] Similar to the unknown task training module 102, the unknown task processing module 103, the unknown task strategy search module 104, and the unknown task preprocessing module 105 may be implemented by hardware or by software.

[0233] When implemented by software, the unknown task processing module 103, the unknown task policy search module 104, and the unknown task preprocessing module 105 may be an application program running on a computing device (such as a server). For example, the unknown task processing module 103, the unknown task policy search module 104, and the unknown task preprocessing module 105 may be a computing engine running on a computing device. The application program may be provided to users in the form of a virtualized service.

[0234] When implemented by hardware, the unknown task processing module 103, the unknown task strategy search module 104, and the unknown task preprocessing module 105 may include at least a computing device, such as a server, etc. Alternatively, the unknown task processing module 103, the unknown task strategy search module 104, and the unknown task preprocessing module 105 may also be implemented by an application-specific integrated circuit ASIC or a programmable logic device PLD.

[0235] In some possible implementations, the unknown task strategy search module 104 is specifically used to:

[0236] Processing the original samples in the validation set by using a sample processing strategy to obtain enhanced samples, wherein the original samples include the second samples in the unknown task samples;

[0237] A preprocessing strategy is obtained according to the inference results of the initial task model on the original sample and the enhanced sample, wherein the preprocessing strategy includes a sample processing strategy in which a second performance value of the inference on the enhanced sample is greater than a first performance value of the inference on the original sample, and the performance value is an indicator value of a performance evaluation indicator determined according to the task type.

[0238] In some possible implementations, the unknown task strategy search module 104 is further configured to:

[0239] According to the post-processing method corresponding to the task type, the reasoning results of the original sample and the enhanced sample are fused, and the fused reasoning results are used to determine the second performance value.

[0240] In some possible implementations, the unknown task strategy search module 104 is further configured to:

[0241] According to the sample processing strategy, the samples in the test set are processed to obtain processed test samples;

[0242] The strategy reliability is determined according to the inference result of the initial task model on the processed test samples and the labeling information of the samples in the test set.

[0243] In some possible implementations, the sample processing strategy is obtained by sampling from a search space, and the interaction module is further used to:

[0244] Receiving the user-defined sample processing strategy;

[0245] The unknown task strategy search module 104 is also used for:

[0246] The user-defined sample processing strategy is added to the search space.

[0247] The present application also provides a computing device 800. Figure 8 As shown, the computing device 800 includes: a bus 802, a processor 804, a memory 806, and a communication interface 808. The processor 804, the memory 806, and the communication interface 808 communicate through the bus 802. The computing device 800 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 800.

[0248] The bus 802 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 The bus 802 may include a path for transmitting information between various components of the computing device 800 (eg, the memory 806, the processor 804, and the communication interface 808).

[0249] The processor 804 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0250] The memory 806 may include a volatile memory, such as a random access memory (RAM). The memory 806 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid state drive (SSD). The memory 806 stores executable program code, and the processor 804 executes the executable program code to implement the aforementioned task processing method. Specifically, the memory 806 stores instructions for the task processing system 10 to execute the task processing method.

[0251] The communication interface 808 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 800 and other devices or communication networks.

[0252] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0253] like Fig. 9As shown, the computing device cluster includes at least one computing device 800. The memory 806 in one or more computing devices 800 in the computing device cluster may store the same task processing system 10 for executing instructions of the task processing method.

[0254] In some possible implementations, one or more computing devices 800 in the computing device cluster may also be used to execute some instructions of the task processing system 10 for executing the task processing method. In other words, a combination of one or more computing devices 800 may jointly execute instructions of the task processing system 10 for executing the task processing method.

[0255] It should be noted that the memory 806 in different computing devices 800 in the computing device cluster may store different instructions for executing partial functions of the task processing system 10 .

[0256] Fig.10 A possible implementation is shown. Fig.10 As shown, computing devices 800A and 800B are connected via a communication interface 808. The memory in computing device 800A stores instructions for executing the functions of the interaction module and the unknown task training module 102. The memory in computing device 800B stores instructions for executing the functions of the unknown task strategy search module 104. Further, computing device 800C can also be connected to computing devices 800A and 800B via a communication interface. The memory in computing device 800C also stores instructions for executing the functions of the unknown task preprocessing module 105 and the unknown task processing module 103. The memory 806 of computing devices 800A and 800B (which may also include computing device 800C) jointly stores instructions for the task processing system 10 to execute the task processing method.

[0257] Fig.10 The connection mode between the computing device clusters shown can be considered that the task processing method provided by the present application requires a lot of computing power for model training and strategy search. Therefore, it is considered that the functions implemented by the unknown task strategy search module 104 are handed over to the computing device 800B for execution. For similar considerations, the functions implemented by the unknown task preprocessing module 105 and the unknown task processing module 103 are handed over to the computing device 800C for execution.

[0258] It should be understood that Fig.10 The functions of the computing device 800A shown in FIG. 8 may also be completed by multiple computing devices 800. Similarly, the functions of the computing devices 800B and 800C may also be completed by multiple computing devices 800.

[0259] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Fig.11 A possible implementation is shown. Fig.11 As shown, computing devices 800D and 800E are connected via a network. Specifically, the network is connected via the communication interface in each computing device. In this type of possible implementation, the memory 806 in the computing device 800D stores instructions for executing the functions of the interaction module and the unknown task training module 102. At the same time, the memory 806 in the computing device 800E stores instructions for executing the functions of the unknown task strategy search module 104. Furthermore, the computing device 800F can also be connected to the computing devices 800D and 800E via the network. The memory in the computing device 800F also stores instructions for executing the functions of the unknown task preprocessing module 105 and the unknown task processing module 103.

[0260] Fig.11 The connection mode between the computing device clusters shown can be considered that the task processing method provided by the present application requires a lot of computing power for model training and strategy search. Therefore, it is considered that the functions implemented by the unknown task strategy search module 104 are handed over to the computing device 800E for execution. For similar considerations, the functions implemented by the unknown task preprocessing module 105 and the unknown task processing module 103 are handed over to the computing device 800F for execution.

[0261] It should be understood that Fig.11 The functions of the computing device 800D shown in FIG. 800A may also be completed by multiple computing devices 800. Similarly, the functions of the computing devices 800E and 800F may also be completed by multiple computing devices 800.

[0262] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by the computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the above-mentioned task processing method applied to the task processing system 10.

[0263] The embodiment of the present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the above-mentioned task processing method.

[0264] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A task processing method, characterized in that: Applied to a task processing system, the task processing system is used to provide a task model for processing unknown tasks, and the unknown tasks are processed based on the task model. The method includes: Receive the initial task model, known task samples, and unknown task samples input by the user; Inputting at least one task sample into the initial task model for reasoning, and determining a weight update strategy according to the reasoning result; The known task samples and the unknown task samples are input into the initial task model, and the model is trained through the weight updating strategy to obtain the task model.

2. The method according to claim 1, characterized in that The known task samples are samples that are successfully inferred by the initial task model, and the unknown task samples are samples that fail to be inferred by the initial task model.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: Determine a performance evaluation indicator according to the reasoning result; The step of inputting the known task samples and the unknown task samples into the initial task model, performing model training through the weight updating strategy, and obtaining the task model comprises: According to the samples in the training set, iteratively updating the weight of the initial task model through the weight update strategy to obtain a transient model, wherein the training set includes the first sample in the unknown task sample and the known task sample; According to the samples in the validation set, the transient model is subjected to performance evaluation by the performance evaluation index to obtain a performance value, wherein the validation set includes the second sample in the unknown task sample; When the performance value of the transient model reaches a performance threshold or the number of iterations of the initial task model reaches a maximum number, the transient model is output as the task model.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: Determine a model reliability evaluation index according to the inference result; According to the samples in the test set, the reliability of the task model is determined by using the model reliability evaluation index.

5. The method according to any one of claims 1 to 4, characterized in that: Determining the weight update strategy according to the inference result includes: Determine the task type according to the inference result, and determine the weight update strategy corresponding to the task type; The task type includes a classification task, a detection task or a segmentation task. The weight update strategy corresponding to the classification task includes a sample-level update strategy, the weight update strategy corresponding to the detection task includes an instance-level update strategy, and the weight update strategy corresponding to the segmentation task includes a pixel-level update strategy.

6. The method according to claim 5, characterized in that The performance evaluation indicators corresponding to the classification task include the F value, the model reliability evaluation indicators corresponding to the classification task include the sample classification error rate, the performance evaluation indicators corresponding to the detection task include the mean average precision mAP, the model reliability evaluation indicators corresponding to the detection task include the instance regression error rate and the instance classification error rate, the performance evaluation indicators corresponding to the segmentation task include the average intersection-over-union ratio mIOU, and the model reliability evaluation indicators corresponding to the segmentation task include the pixel classification error rate.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Acquire a preprocessing strategy according to the unknown task sample, the initial task model and the task type determined by the reasoning result; Using the preprocessing strategy to enhance the unknown task to be processed; The enhanced unknown task is processed by the task model.

8. The method according to claim 7, characterized in that The obtaining of a preprocessing strategy according to the unknown task sample, the initial task model and the task type determined by the inference result includes: Processing the original samples in the validation set by using a sample processing strategy to obtain enhanced samples, wherein the original samples include the second samples in the unknown task samples; A preprocessing strategy is obtained according to the inference results of the initial task model on the original sample and the enhanced sample, wherein the preprocessing strategy includes a sample processing strategy in which a second performance value of the inference on the enhanced sample is greater than a first performance value of the inference on the original sample, and the performance value is an indicator value of a performance evaluation indicator determined according to the task type.

9. The method according to claim 8, characterized in that The method further comprises: According to the post-processing method corresponding to the task type, the reasoning results of the original sample and the enhanced sample are fused, and the fused reasoning results are used to determine the second performance value.

10. The method according to claim 8, characterized in that The method further comprises: According to the sample processing strategy, the samples in the test set are processed to obtain processed test samples; The strategy reliability is determined according to the inference result of the initial task model on the processed test samples and the labeling information of the samples in the test set.

11. The method according to any one of claims 8 to 10, characterized in that: The sample processing strategy is obtained by sampling from the search space, and the method further includes: Receiving the user-defined sample processing strategy; The user-defined sample processing strategy is added to the search space.

12. A task processing system, characterized in that: The task processing system is used to provide a task model for processing unknown tasks, and processes the unknown tasks based on the task model. The system includes: An interactive module, used to receive the initial task model, known task samples, and unknown task samples input by the user; The unknown task training module is used to input at least one task sample into the initial task model for reasoning, determine the weight update strategy according to the reasoning result, input the known task sample and the unknown task sample into the initial task model, perform model training through the weight update strategy, and obtain the task model.

13. The system according to claim 12, characterized in that The known task samples are samples that are successfully inferred by the initial task model, and the unknown task samples are samples that fail to be inferred by the initial task model.

14. The system according to claim 12 or 13, characterized in that The unknown task training module is also used for: Determine a performance evaluation indicator according to the reasoning result; The unknown task training module is specifically used for: According to the samples in the training set, iteratively updating the weight of the initial task model through the weight update strategy to obtain a transient model, wherein the training set includes the first sample in the unknown task sample and the known task sample; According to the samples in the validation set, the transient model is subjected to performance evaluation by the performance evaluation index to obtain a performance value, wherein the validation set includes the second sample in the unknown task sample; When the performance value of the transient model reaches a performance threshold or the number of iterations of the initial task model reaches a maximum number, the transient model is output as the task model.

15. The system according to any one of claims 12 to 14, characterized in that: The unknown task training module is also used for: Determine a model reliability evaluation index according to the inference result; According to the samples in the test set, the reliability of the task model is determined by using the model reliability evaluation index.

16. The system according to any one of claims 12 to 15, characterized in that The unknown task training module is specifically used for: Determine the task type according to the inference result, and determine the weight update strategy corresponding to the task type; The task type includes a classification task, a detection task or a segmentation task. The weight update strategy corresponding to the classification task includes a sample-level update strategy, the weight update strategy corresponding to the detection task includes an instance-level update strategy, and the weight update strategy corresponding to the segmentation task includes a pixel-level update strategy.

17. The system according to claim 16, characterized in that The performance evaluation indicators corresponding to the classification task include the F value, the model reliability evaluation indicators corresponding to the classification task include the sample classification error rate, the performance evaluation indicators corresponding to the detection task include the mean average precision mAP, the model reliability evaluation indicators corresponding to the detection task include the instance regression error rate and the instance classification error rate, the performance evaluation indicators corresponding to the segmentation task include the average intersection-over-union ratio mIOU, and the model reliability evaluation indicators corresponding to the segmentation task include the pixel classification error rate.

18. The system according to any one of claims 12 to 17, characterized in that The system further comprises: An unknown task strategy search module, used to obtain a preprocessing strategy according to the unknown task sample, the initial task model and the task type determined by the reasoning result; An unknown task preprocessing module, used for enhancing the unknown task to be processed using the preprocessing strategy; The unknown task processing module is used to process the enhanced unknown task through the task model.

19. The system according to claim 18, characterized in that The unknown task strategy search module is specifically used for: Processing the original samples in the validation set by using a sample processing strategy to obtain enhanced samples, wherein the original samples include the second samples in the unknown task samples; A preprocessing strategy is obtained according to the inference results of the initial task model on the original sample and the enhanced sample, wherein the preprocessing strategy includes a sample processing strategy in which a second performance value of the inference on the enhanced sample is greater than a first performance value of the inference on the original sample, and the performance value is an indicator value of a performance evaluation indicator determined according to the task type.

20. The system according to claim 19, characterized in that The unknown task strategy search module is also used for: According to the post-processing method corresponding to the task type, the reasoning results of the original sample and the enhanced sample are fused, and the fused reasoning results are used to determine the second performance value.

21. The system according to claim 19, characterized in that The unknown task strategy search module is also used for: According to the sample processing strategy, the samples in the test set are processed to obtain processed test samples; The strategy reliability is determined according to the inference result of the initial task model on the processed test samples and the labeling information of the samples in the test set.

22. The system according to any one of claims 19 to 21, characterized in that The sample processing strategy is obtained by sampling from the search space, and the interaction module is further used for: Receiving the user-defined sample processing strategy; The unknown task strategy search module is also used for: The user-defined sample processing strategy is added to the search space.

23. A computing device cluster, characterized in that: The computing device cluster includes at least one computing device, and the at least one computing device includes at least one processor and at least one memory, wherein the at least one memory stores computer-readable instructions; the at least one processor executes the computer-readable instructions so that the computing device cluster executes the method as described in any one of claims 1 to 11.

24. A computer-readable storage medium, characterized in that: The method comprises computer-readable instructions; the computer-readable instructions are used to implement the method according to any one of claims 1 to 11.

Citation Information

Cited By

  • Task starting method and device and electronic equipment

    CN121071869A