To provide a device, a dataset and a computer-implemented method for testing
automated reasoning, in particular, automated tabular data reasoning, for determining an answer to a question based on a table including contents of cells in a tabular structure.SOLUTION: The method includes: providing a table, a question, and an answer (300); determining a prediction for an answer by
automated reasoning based on contents of cells of the table and the question (304-2, 306-2, 308-2); and determining, depending on a difference between the prediction and the answer, at least two of a robustness metric of
automated reasoning, in particular, automated tabular data reasoning, with respect to changes in the tabular structure, an attention metric of automated reasoning, in particular, automated tabular data reasoning, with respect to at least one
cell of the table, and a robustness metric with respect to changes in numerical contents of at least one
cell of the table (304-3, 306-3, 308-3).SELECTED DRAWING: Figure 3