An expert selection method for human-machine collaboration oriented information gain and dynamic confusion matrix

By predicting the distribution of expert labels and generating a dynamic confusion matrix based on the current sample features in a human-machine collaborative system, the problem of low efficiency in expert subset selection in existing technologies is solved. This enables the accurate selection of expert subsets that can improve team decision-making performance when expert labels cannot be obtained in advance, thereby improving decision-making accuracy and resource utilization efficiency.

CN122221033APending Publication Date: 2026-06-16NORTHWESTERN POLYTECHNICAL UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202610453300.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-04-08
Publication Date
2026-06-16

AI Technical Summary

Technical Problem

Existing methods for selecting human expert subsets are inefficient and cannot dynamically assess expert capabilities, leading to wasted human resources and reduced prediction accuracy. Especially when it is impossible to obtain expert annotations for the current samples in advance, how to accurately and dynamically assess expert capabilities and select the expert subset that can maximize the team's decision-making performance is a technical problem that urgently needs to be solved.

Method used

By predicting the distribution of expert labels based on the current sample features, a dynamic confusion matrix is ​​generated. Combined with the initial predicted probability distribution of the target AI model, the optimal subset of experts is iteratively selected, and the prediction results are fused with the actual annotations to achieve dynamic evaluation and efficient selection.

Benefits of technology

In situations where expert annotations cannot be obtained in advance, accurately assessing expert capabilities and efficiently selecting a subset of experts that can maximize the team's decision-making performance improves the efficiency of expert resource utilization and the accuracy of decision-making.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122221033A_ABST
    Figure CN122221033A_ABST
Patent Text Reader

Abstract

This application relates to the field of human-machine collaboration technology, and particularly to an expert selection method based on information gain and dynamic confusion matrix for human-machine collaboration. The method includes: predicting the label prediction distribution for each expert in the expert pool for the current sample; for each expert in the expert pool, generating a dynamic confusion matrix for the current sample based on their corresponding label prediction distribution and static confusion matrix; iteratively selecting an optimal subset of experts from the expert pool based on the initial prediction probability distribution of the target AI model for the current sample and the dynamic confusion matrices of each expert, with the goal of minimizing decision uncertainty; and fusing the prediction results with the actual labels to obtain the final decision result for the current sample. This method can accurately and dynamically evaluate expert capabilities and efficiently select an expert subset that maximizes the team's decision-making performance when the experts' labels for the current sample cannot be obtained in advance.
Need to check novelty before this filing date? Find Prior Art