Condition Definition Using Image-Text Feature Fusion for Damage Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for detecting damage in infrastructure such as bridges and tunnels, like CLIP and DualCoOp, are sensitive to slight differences in summary texts, leading to inconsistent damage classification and oversights in visual inspections.
Innovation Solution
A condition definition apparatus and method that integrates image and text features using a neural network to generate an integrated feature vector, combining image, summary, and detailed text data for accurate damage classification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If CLIP or DualCoOp methods are used for damage classification, then image processing capability is improved, but classification consistency deteriorates due to sensitivity to slight text differences
Solution Approach 1:
The patent combines image data, summary text data, and detailed text data into a unified processing framework. Multiple data sources are integrated to form a comprehensive feature representation, where image features, summary features, and detailed features are merged to achieve consistent and accurate damage classification that is not sensitive to text variations
Solution Approach 2:
The patent introduces an intermediary processing layer that transforms multiple data sources into standardized feature vectors. This intermediary layer includes feature extraction modules that convert images, summaries, and detailed texts into comparable feature representations, mediating between different data types and the classification system
2Adaptability or versatility
If visual inspection is performed by human inspectors, then flexibility in assessment is improved, but detection accuracy deteriorates due to individual differences
Solution Approach 1:
The system performs self-assessment by automatically processing inspection data without human intervention. The machine learning model independently evaluates damage conditions by processing images and text data, eliminating the need for human inspectors while maintaining consistent and accurate detection across all assessments
Solution Approach 2:
The patent replaces the mechanical human inspection process with an automated computational system. Human visual inspection is substituted with machine-based image processing and text analysis, where neural networks and feature extraction algorithms perform the assessment functions previously done by human inspectors
Data Source
AI summary
A condition definition apparatus includes: accepting first inspection object data showing an image of a first inspection object; summary text data explaining a summary of condition of the inspection object in text, and detailed text data explaining details of condition of the inspection object in text, generating an image feature vector showing features of the image shown by the first inspection object data from the accepted first inspection object data, generating a summary feature vector showing summary of features of the condition of the inspection object shown by the summary text data from the accepted summary text data, generating an integrated feature vector by integrating the generated summary feature vector and generating the detailed feature vector and inspection object class data used to define the condition of the inspection object from the integrated feature vector.