Multimodal Plant Database Search for Operating Time Intervals
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current keyword-based search methods in plant databases are limited to text data and fail to incorporate multimodal aspects, requiring manual effort and domain knowledge to identify relevant time intervals with interesting events.
Innovation Solution
Implementing an AI/ML search component that processes multimodal data, including text, images, and time series, to find operating characteristics and their time intervals within a plant database, using a generative AI/ML search component and a joint embedding space to contextualize data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If keyword-based search methods are used, then text data can be searched, but multimodal aspects cannot be incorporated and manual effort is required
Solution Approach 1:
The search system is enhanced to handle multiple data modalities (text, images, time series, audio) through a unified multimodal embedding space that can process and retrieve across different data types, making the search system universal and multi-functional rather than limited to text-only keyword search
Solution Approach 2:
A joint embedding space acts as an intermediary layer that translates different modalities (text, images, time series) into a common representation space, enabling the search system to process multimodal data without requiring separate processing pipelines for each data type
2Loss of information
If manual search in multiple data types is performed, then comprehensive information can be gathered, but intensive manual effort and domain know-how are required
Solution Approach 1:
The system performs self-service by automatically retrieving and correlating information across multiple data modalities without requiring manual intervention. The AI model autonomously searches through text, images, time series, and other data types to compile comprehensive results, eliminating the need for operators to manually query each data source
3Productivity
If partial search in specific data types is conducted, then search speed can be maintained, but comprehensive plant situation understanding is limited
Solution Approach 1:
The system merges search results from multiple data modalities (text logs, images, time series data, audio recordings) into a unified comprehensive result set. By combining these different data sources through the joint embedding space, the system maintains search speed while providing holistic contextual understanding of plant situations
Data Source
AI summary
A method for obtaining a search result for a search query within a database system of a plant, the database system configured for storing multimodal operating data of the plant, the method comprising: obtaining the search query indicative of an operating characteristic in the operating of the plant to be searched within the database system; executing the search query by an artificial intelligence/machine learning, AI/ML, search component within the database system for finding the searched operating characteristic and time intervals of the operating characteristic among different data modalities of the multimodal data therein; and obtaining a search result indicative of the operating characteristic and its time intervals found by the execution of the search query by the AI/ML search component.