Multimodal Congruence Detection for Speech and Body Language

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing technologies fail to detect congruence or incongruence between a person's body language and speech, which is crucial for confirming or disconfirming interviewer intuition and guiding appropriate communication strategies, especially in live interviews.

Innovation Solution

A method and system using a video recording device to capture visual and audio cues, dividing the recording into sequences, detecting and rating visual and audio cues, and comparing these ratings to provide congruence or incongruence indicators through a self-learning machine trained on diverse speech and body language parameters.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of information

If existing technologies are used to detect speech content, then verbal information can be obtained, but the congruence between body language and speech cannot be assessed

Engineering Contradiction:

Improvecongruence informationVSAvoidsystem complexity

Core Design Contradiction:

Loss of informationVSDevice complexity

Solution Approach 1:

The patent combines multiple analysis components (facial expression analysis, body language analysis, speech content analysis) into a single integrated system that processes video and audio inputs simultaneously to detect congruence between verbal and non-verbal cues

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The system performs multiple functions using a unified approach: it analyzes facial expressions, body language, and speech content while also determining congruence, providing a comprehensive behavioral assessment tool applicable to various fields including law enforcement, human resources, and psychology

Inventive Principle:

Principle #6Universality (Multi-functionality)

2Measurement precision

If manual analysis of body language and speech is performed, then congruence can be assessed, but the process is time-consuming and subjective

Engineering Contradiction:

Improvecongruence detection accuracyVSAvoidanalysis time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent replaces manual human analysis with an automated computer-based system that uses image processing, audio processing, and pattern recognition algorithms to objectively detect and analyze congruence between body language and speech, eliminating subjectivity and reducing analysis time

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The system provides real-time or near-real-time feedback on congruence detection results, allowing immediate assessment of whether verbal and non-verbal cues are consistent, which can be used for training or decision-making purposes

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS12620261B2System and method for reading and analysing behaviour including verbal, body language and facial expressions in order to determine a person's congruence

Publication Date: 2026.05.05 PLENIUM AG

AI summary

According to the invention, is provided a data processing system for determining congruence or incongruence between the body language and the Speech of a person, comprising a self-learning machine, such as a neutralneural network, arranged for receiving as input a dataset including: approved data of a collection of analysed Speeches of persons, said approved data comprising for each analysed Speech: * a set of video sequences, comprising audio sequences and visual sequences, each audio sequence corresponding to one visual sequence, and * an approved congruence indicator for each of said video sequence—said self-learning machine being trained so that the data processing system is able to deliver as output a congruence indicator.