A method and a
system for generating an explainable prediction of an emotion associated with a vocal sample are disclosed. The method includes receiving, by a
processing device, a vector representation (z,) of an initial prediction (y0) of the emotion associated with the vocal sample (x), a counterfactual synthetic vocal sample (xY) associated with the vocal sample (x) and an alternate emotion (y) different from the initial prediction (y0) of the emotion, a vector representation (z,) of an emotion prediction (y0) associated with the counterfactual synthetic vocal sample (xy), vocal cue information (cy, cy) associated with the vocal sample (x) and the counterfactual synthetic vocal sample (xY) and attribution explanation information (iVc7) associated with relative importance of the vocal cue information (cy, cy) in prediction of the emotion. The method also includes determining, using the
processing device, numeric cue differences (cyy) between the vocal cue information (cy) associated with the vocal sample (x) and the vocal cue information (cy) associated with the counterfactual synthetic vocal sample (xy), generating, using the
processing device, cue difference relations information (r{
circumflex over ( )}) based on the attribution explanation information (iv{
circumflex over ( )}7), the numeric cue differences (cyy), the vector representation (z,) of the initial prediction (y0) and the vector representation (z,) of the emotion prediction (y0) associated with the counterfactual synthetic vocal sample (xY) using a first neural network (Mr), generating, using the processing device, a final prediction (y) of the emotion based on the numeric cue differences (cyy), the vector representation (z,) of the initial prediction (y0) and the vector representation (z,) of the emotion prediction (y0) associated with the counterfactual synthetic vocal sample (xY) using a second neural network (My), and generating, using the processing device, the explainable prediction of the emotion associated with the vocal sample (x) based on at least the counterfactual synthetic vocal sample (xY), the final prediction (y) of the emotion and the cue difference relations information (r{
circumflex over ( )}).