Computer-implemented system for semantic search in a corpus of standards for the construction and trades using vector embeddings

Abstract

A computer-implemented search system (10) for building standards, comprising a database (20) in which a corpus of standards (30) is stored, wherein each standard of the corpus of standards (30) is stored as a data object (40) with at least one standard number and a title, and a query interface (280) configured to receive a natural language search query (70), characterized in that a) at least one numerical vector embedding (50) is stored in the database (20) for each standard of the corpus of standards (30), which represents the content of the respective standard as a vector in a multidimensional vector space; b) the system comprises an embedding module (60) configured to transform the natural language search query (70) into a query vector embedding (80) of the same vector space by means of an embedding model (170);c) the system includes a vector search unit (100) which is configured to perform a similarity search in the vector space, in particular based on cosine similarity, between the query vector embedding (80) and the stored vector embeddings (50) and thereby to find semantically similar norms independently of a literal match with the query query (70);d) the system comprises a hybrid search module (90) configured to process the search query (70) using at least three different search methods, one of which is the similarity search in the vector space according to feature c), wherein the hybrid search module (90) comprises at least the following as non-embedding-based search methods: a full-text search unit (110) configured to perform a language-specific full-text search on fields of the data objects (40), and a standard number recognition unit (120) configured to extract standard numbers of at least three different technical regulations, in particular the DIN, VDE and DVGW regulations, from the search query (70) using regular expressions;e) the hybrid search module (90), in particular the standard number recognition unit (120), is configured to expand the search query (70) domain-specifically using a synonym expansion table (260) with construction-specific synonym assignments, whereby the expansion targets of the synonym assignments include both standard technical vocabulary and standard numbers, so that the non-embedding-based search methods also provide hits for everyday language search terms whose wording does not occur in the respective standard;f) the hybrid search module (90) comprises a score fusion unit (140) which has several different weighting schemes and is configured to merge the hits of the search methods into a uniform result list (150) ordered according to an overall relevance value by means of weighted score fusion and to select the weighting scheme to be applied to a hit depending on the identity of the combination of those search methods that found the hit, i.e. depending on which of the search methods found the hit; and g) each data object (40) has a trade assignment (160) to at least one craft trade from a predefined list of trades, such that the standards corpus (30) is categorized according to craft trades.