The invention discloses a risk
feature design method based on
multiple modes, and aims to solve the problems of single
feature extraction dimension, simple fusion method, poor model adaptability and insufficient
interpretability in the existing
risk assessment technology. According to the method, multi-
modal data such as a living picture, a face-to-face audit video and a marketing voice call of a user are collected at first, and then targeted preprocessing is conducted, specifically, the background of the living picture is extracted,
jitter of the face-to-face audit video is eliminated, a
timestamp is aligned, and
noise reduction and mute detection are conducted on the voice call. Then, extracting multi-dimensional features such as face attributes, expression changes and emotions from the preprocessed data, and after Z-
score standardization, fusing the features through a cross-
modal attention mechanism of a multi-
modal Transform architecture and a graph neural network; and finally, a
reinforcement learning model containing a pre-training
large model encoder, a strategy network and a composite reward function is utilized to output risk scores and grades, and decision transparency is guaranteed in combination with
interpretability design.