The invention discloses a court trial
verification method based on a multi-
modal Transform, and the method comprises the following steps: collecting multi-
modal information data, and carrying out the preprocessing and
feature extraction; inputting the data to a
modal encoder for dimension adjustment and
standardization processing; inputting to a fusion module of the TriMERT model to realize
semantic alignment and
feature fusion; inputting the data to a reasoning module, and carrying out multi-layer
feature coding and context modeling; comparing with the vocabulary semantic features, generating
verification information, and integrating and judging; calculating a modal contribution
score, and performing feature weighting and
key frame labeling on each modal; and integrating the
verification judgment result and the reasoning path indication, and displaying in a structured report form. According to the method, a multi-mode Transform modeling method is adopted, court hearing semantic verification and reasoning
path generation are achieved, and the method has the advantages of being high in fusion degree, accurate in recognition and high in
interpretability.