Chained Confidence Scoring for Automated Loan Audit

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current loan audit processes rely heavily on human experience and judgment, leading to inconsistencies, labor-intensive methods, and incomplete analysis, as they often focus on a representative sample of loans rather than the entire portfolio, introducing risk and inefficiency.

Innovation Solution

An automated loan audit system employing chained confidence scoring, which captures and quantifies confidence values for each step in the loan analysis, using active learning to generate training data and calibrate confidence scores, allowing for the automated processing of commercial mortgage loans and reducing dependency on individual auditors.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If automated processing is implemented, then productivity and consistency are improved, but measurement precision and reliability may deteriorate due to lack of human judgment

Engineering Contradiction:

Improveloan audit processing speedVSAvoidloan risk assessment accuracy

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The loan audit process is divided into discrete cognitive processing tasks (data extraction, document classification, risk factor identification, etc.), with each task assigned a confidence value. This segmentation allows automated processing while maintaining quality control through individual task validation.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system implements feedback loops where user corrections and case feedback are captured and used to retrain the machine learning models. This continuous feedback mechanism improves measurement precision over time while maintaining high productivity through automated processing.

Inventive Principle:

Principle #23Feedback

2Measurement precision

If human auditors review entire portfolios, then measurement precision is improved, but productivity deteriorates due to resource limitations

Engineering Contradiction:

Improveloan risk assessment accuracyVSAvoidportfolio analysis capacity

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The system introduces an intermediary layered approach where automated cognitive processing tasks with confidence scoring serve as an intermediate layer between complete automation and full human review. This allows the system to process entire portfolios efficiently while maintaining accuracy through selective human intervention on low-confidence cases.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system dynamically adjusts the confidence threshold parameter to balance productivity and precision. By changing this parameter, the system can optimize between processing speed (higher threshold) and assessment accuracy (lower threshold), allowing flexible adaptation to different portfolio sizes and risk tolerances.

Inventive Principle:

Principle #35Parameter changes

3Productivity

If representative sampling is used, then productivity is improved by reducing workload, but reliability deteriorates due to incomplete portfolio analysis

Engineering Contradiction:

Improveaudit efficiencyVSAvoidportfolio risk assessment completeness

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The system performs partial automated review of all loans with confidence scoring, then applies excessive action by allowing selective human review of specific high-risk or low-confidence cases. This approach provides more comprehensive coverage than traditional sampling while maintaining efficiency through automated processing of clear-cut cases.

Inventive Principle:

Principle #16Partial or excessive action

4Reliability

If multiple human reviewers are used to resolve differences, then reliability is improved through cross-validation, but productivity deteriorates due to increased labor requirements

Engineering Contradiction:

Improveassessment consistencyVSAvoiddispute resolution time

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The system replaces the mechanical process of multiple human reviewers with automated machine learning models that provide consistent, repeatable assessments. The chained confidence scoring mechanism ensures reliability through mathematical consistency rather than human consensus, dramatically improving productivity while maintaining assessment consistency.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS11010832B2Loan audit system and method with chained confidence scoring

Publication Date: 2021.05.18 KPMG LLP

AI summary

The invention relates to a computer-implemented system and method for grading of a loan using chained confidence scoring. The method may comprise the steps of: scanning documents within a credit file for the loan, extracting attributes from the scanned documents and from electronic documents in the credit file, calculating a plurality of calculated attributes based on the extracted attributes, calculating a loan risk rating based on the calculated attributes and the extracted attributes, calculating an aggregated confidence value associated with the calculated loan risk rating, and enabling a user to modify the loan risk rating, the aggregated confidence value, and a number of chained confidence values. The confidence values input by the user are used as training data to train a chained confidence model and the chained confidence model is used to calculate the aggregated confidence value in connection with the automated grading of a loan.