Contradiction detection between texts

Abstract

The invention relates to the examination of texts for content contradictions. In order to reduce the effort required for the examination, the invention provides that content-related similar texts are grouped (16) and the grouped texts are examined for content contradictions (18, 24, 30).

Description

[0001] The invention relates to the detection of content-related contradictions between several texts.

[0002] Texts from large collections often need to be checked for inconsistencies. For example, technical documentation or other documents must be checked for contradictions to obtain or maintain quality certification. Large projects also generate a multitude of texts describing the project. Contradictions in these texts can lead to costly problems during project implementation. Specifications, standards, requirements from component specifications for technical components (such as automotive components), system specifications for technical systems, or internal company standards are also contained in numerous texts. These texts must be checked for inconsistencies to avoid or at least minimize errors. However, with large collections of texts, it is virtually impossible to perform this inconsistency check manually.Therefore, contradiction checking is often automated, and recently artificial intelligence (AI), such as a large language model (LLM, for example, a generative pre-trained transformer or GPT), has also been used for this purpose. However, when a text collection is loaded into a large language model to check its texts for contradictions, the check can require significant resources, especially processor and memory resources. The more texts that need to be checked for contradictions and / or the longer the texts to be checked for contradictions are, the greater the resources required and the associated costs.For example, using a brute-force approach, where all possible pairings of statements in the texts are checked, leads to a quadratically increasing effort for n statements and thus quickly to enormous computing and API costs, especially if current Large Language models are used for the check.

[0003] The purpose of the invention is to check several texts for content contradictions in the most cost-effective way possible and with the least possible expenditure of resources.

[0004] The invention is defined by the features of the independent claim. Advantageous further developments and embodiments are the subject of the dependent claims.

[0005] The task is solved by an artificial intelligence (AI)-based method for detecting content contradictions between multiple texts. In this method, each text is converted into a semantic vector. At least one property of each vector is determined. Vectors whose properties are more similar to each other than the properties of at least one other vector are assigned to a property group. For texts converted into vectors assigned to this property group, an AI-supported test is then performed to detect content contradictions. For example, the AI-supported test for detecting contradictions is only performed on texts converted into vectors assigned to this property group.

[0006] Furthermore, the problem is solved by a device for detecting content contradictions between several texts, wherein the device has a processor, a memory containing instructions which is connected to the processor for data transmission, and an interface connected to the processor for data transmission for receiving several texts, wherein the processor, when it executes the instructions contained in the memory, receives the several texts via the interface and then executes the method according to the invention.

[0007] Furthermore, the problem is solved by a computer program product comprising instructions which, when the program is executed by a computer with a processor and a memory containing multiple texts, cause the computer to execute the method according to the invention.

[0008] Furthermore, the problem is solved by a computer-readable storage medium comprising instructions which, when executed by a computer, cause it to execute the method according to the invention.

[0009] By grouping the texts into attribute groups, the content of the texts to be examined is more closely related, so that texts with similar content are checked for contradictions. This avoids unnecessarily including texts that are more distant in content in the examination. Furthermore, the thematic proximity of the texts makes the examination easier to understand, which can make adverse effects of the use of artificial intelligence, such as so-called hallucinations, more readily apparent.

[0010] For example, specifications containing texts for the same or similar components can be grouped into one property group, whereas specifications containing texts for other components are not assigned to the property group and are grouped, for example, in a different property group. The components could be, for example, trunk lids, and the other components could be, for example, hoods. Trunk lids usually have an externally accessible opening handle, but hoods generally do not. If the AI ​​finds such an opening handle in one of the hood specifications, this indicates a contradiction between the hood specifications.If hoods and trunk lids were checked together for inconsistencies, the AI ​​might not be able to recognize that a hood specification with a handle contradicts other hood specifications, since similar trunk lids also have such a handle, which is specified in their respective trunk lid specifications. Even if the AI ​​were to detect this inconsistency, it would be comparatively costly, as it would have to check both hood and trunk lid specifications. Furthermore, it would be difficult for a human to understand why the handle constitutes a contradiction if the AI ​​checks hood and trunk lid specifications together.

[0011] In one embodiment of the method, the vector property is the magnitude of the vector or its direction.

[0012] In one implementation of the method, the texts are transformed into semantic vectors (e.g., vectors with 384- or 768-dimensional embedding) by an AI, for example, a pre-trained language model such as a Sentence Transformer. The resulting set of vectors or embeddings can represent the content of the texts in a continuous vector space. Subsequently, the vectors or embeddings can be subdivided into groups of similar statements using an unsupervised clustering algorithm, such as K-means or HDBSCAN. The number of groups or clusters can be dynamically limited, for example, to at least 5%, up to 10%, up to 15%, up to 20%, or up to 25% of the number of texts or the number of statements contained in the texts. This allows for a meaningful, but not overly granular, grouping.For each group / cluster, the AI, for example a large language model (e.g. GPT-4), can generate a short description summarizing all statements of the group / cluster.

[0013] In one embodiment of the procedure, a property center is determined for the property group. Vectors whose properties are at most a predetermined distance from the property center are selected. For texts whose vectors have been selected, AI-supported inconsistency testing is performed. For example, AI-supported inconsistency testing is only performed for texts whose vectors have been selected. This allows for more accurate content-based grouping of texts, further improving the efficiency and traceability of the testing. The vector property can be the vector's magnitude or its direction.

[0014] The property center of the property group can be an averaged vector formed from the vectors of the property group. At least one property of the averaged vector can be an averaged property of the vectors of the property group. The averaged property of the averaged vector can correspond to the averaged direction and / or the averaged magnitude of the vectors of the property group. The predetermined maximum distance of the vectors from the property center can represent a similarity between the vectors. The closer the vectors are to the property center, the greater the similarity. Similarity can also serve as a threshold, whereby the similarity of the vectors must at least equal the similarity threshold for the texts to be selected for AI-assisted inconsistency testing.The predetermined similarity threshold can be a cosine similarity of at least 0.5, at least 0.6, at least 0.7, at least 0.75, at least 0.8, at least 0.9, at least 0.95 or greater.

[0015] In one embodiment of the procedure, the vectors are assigned to different property groups. The AI-supported check for detecting contradictions between texts that have been converted into vectors assigned to one property group is performed separately from the AI-supported check for detecting contradictions between texts whose vectors are assigned to a different property group. This further improves the efficiency and traceability of the check. Texts from the property groups can be selected for the AI-supported contradiction check based on the distance of their vectors from the property center of the respective property group.

[0016] If the vectors have been assigned to different property groups, a property center can be determined for each property group, either additionally or alternatively. AI-assisted testing for detecting contradictions between texts converted into vectors assigned to one property group can be performed together with texts whose vectors are assigned to a different property group, provided the distance between the property centers of two property groups is less than a predetermined threshold. For example, the predetermined threshold could correspond to the predetermined similarity threshold.

[0017] The center of each property group can be an averaged vector formed from the vectors of that property group. At least one property of the averaged vector can be an averaged property of the vectors of that property group. The distance between the centers of the properties, which must be less than the predetermined threshold, can represent a similarity between the centers. The closer the centers of the properties are, the greater the similarity. Similarity can also serve as a threshold, whereby the similarity of the centers of the properties must be at least equal to the similarity threshold for the AI-assisted check to detect contradictions between texts from both property groups to be performed together. The predetermined similarity threshold can correspond to a cosine similarity of at least 0.5, at least 0.75, or up to 1.The texts of the property groups can be selected for AI-supported testing to detect contradictions based on the distance of their vectors to the property center of the respective property group.

[0018] Contradiction checking can be performed using pairs of statements contained in the texts that belong either to the same group / cluster or to two groups / clusters identified as sufficiently similar. Two statements can be provided to the AI ​​at a time. In response to a corresponding prompt, the AI ​​can explicitly evaluate and classify semantic contradictions between the two texts or between a sentence in one text and a sentence in the other ("contradiction" vs. "no contradiction," or similar, depending on the use case and the prompting of the AI ​​called via API). The results can be aggregated and displayed, ideally with further information about the group (for example, an AI-generated short description), so that a user can understand the identified contradictory statements within their thematic context.

[0019] For example, the technical implementation of the contradiction check can be done in Python, using libraries such as HuggingFace Transformers for embedding models, scikit-learn for clustering, NumPy for vector computation, and OpenAL or Azure APIs for LLM queries. Optimizations such as parallelizing LLM calls or caching known embedding clusters can further improve performance.

[0020] According to the invention, the contradiction check using AI can be performed within a group of properties or between statements from similar groups of properties. This allows the number of necessary AI calls to be drastically reduced to O(n*k) (with k < n).

[0021] The AI ​​can be trained to detect contradictions between texts, for example, by using sample texts and optionally with the help of a human trainer or another AI. The AI ​​can be trained to group texts, for example, by using sample texts and optionally with the help of a human trainer or another AI. In particular, the AI ​​can be trained to identify similarities between semantic vectors, for example, by using sample vectors and optionally with the help of a human trainer or AI trainer. The AI ​​can be trained to convert at least selected sentences or sentence fragments of a text, or the entire text, into a semantic vector, for example, by using sample texts and sample vectors, and optionally with the help of a human trainer or another AI.

[0022] The device according to the invention can be configured with one or more computing units. The device can be a computer, and in particular any common type of computing unit, such as a desktop computer, a laptop, a server or server cluster, a system-on-a-chip, or the like. Due to the high computing demands of large language models, high-performance computing clusters are particularly suitable for this purpose. The individual logical modules of the device according to the invention can also be distributed across several physical computing units. The device according to the invention can have corresponding interfaces for receiving and transmitting data.The device can be configured to access a computer-readable storage medium containing at least one computer program. The execution of this program by a processor of the data sharing system causes the system to execute corresponding process steps. The interface can be logical or physical. Texts can be received and / or checked in at least one common document format, such as PDF, DOCX, or XML. The texts to be checked for inconsistencies can each be contained within a single document. Alternatively or additionally, the texts to be checked for inconsistencies can be different sections of a document. The computer program can be an integral part of the device, provided as a separate data carrier, or available as a download.

[0023] The invention can therefore drastically reduce the necessary contradiction checks and thus the computation time, costs, and latency, especially when using expensive LLM APIs such as those from OpenAI (GPT models). Through grouping, the invention can improve the scalability of contradiction detection in large text collections without overlooking relevant semantic conflicts. The systematic grouping of similar statements can increase the traceability of the analysis, as detected contradictions can be output within a clearly defined thematic context. The method can be used flexibly and independently of the content of the specific texts and can be applied to any textual data, as well as to other LLM-based tasks beyond pure contradiction checking.

[0024] Further advantages, features, and details will become apparent from the following description, in which – possibly with reference to the drawings – at least one embodiment is described in detail. Identical, similar, and / or functionally equivalent parts are identified by the same reference numerals.

[0025] They show: Fig. 1 an embodiment of the method according to the invention, and Fig. 2 a further embodiment of the method according to the invention, Fig. 3 a further embodiment of the method according to the invention, and Fig. 4 an embodiment of the device according to the invention.

[0026] Fig. Figure 1 shows an embodiment of the inventive method 10 for detecting content contradictions between several texts. In a process step 12, the texts are each converted into a semantic vector. In the following process step 14, at least one property of each of the vectors is determined. In the subsequent process step 16, vectors whose properties are more similar to each other than the properties of at least one other vector are assigned to a property group. Different vectors can be assigned to different property groups. This is followed by process step 18, in which an AI-supported test for detecting content contradictions is performed on the texts, and possibly only on the texts that were converted into vectors assigned to the property group.

[0027] Fig. Figure 2 shows a further embodiment of the inventive method 10 for detecting content contradictions between several texts. Method step 20 can follow method step 16. In method step 20, a property center of the property group can be determined. In the following method step 22, vectors whose properties have a maximum predetermined distance from the property center can be selected. For texts of the property group whose vectors have been selected, the AI-supported test for contradiction detection can be carried out in the following method step 24. Optionally, the AI-supported test for contradiction detection can be carried out in the following method step 24 only for texts of the property group whose vectors have been selected.

[0028] Fig. Figure 3 shows a further embodiment of the inventive method 10 for detecting content contradictions between several texts. Method step 26 can follow method step 16. In method step 26, a property center of each of the property groups can be determined. In the subsequent method step 28, the distance between the property centers can be determined.

[0029] In the following process step 30, the AI-supported check for detecting contradictions between texts that have been converted into vectors assigned to one of the property groups, together with texts whose vectors are assigned to another property group, can be carried out if the distance between the property centers of two property groups is less than a predetermined threshold. The property groups whose property centers are less than the predetermined threshold can be merged into a single property group.

[0030] Alternatively or additionally, in process step 30, the AI-supported check for detecting contradictions between texts that have been converted into vectors assigned to one of the property groups can be carried out separately from the AI-supported check for detecting contradictions between texts whose vectors are assigned to another of the property groups.

[0031] Fig.Figure 4 shows an embodiment of the device 32 according to the invention for detecting content contradictions between several texts. The device 32 comprises a processor 34, a memory 36 containing instructions and connected to the processor 34 for data transmission, and an interface 38 connected to the processor 34 for receiving several texts. When the processor 34 executes the instructions contained in the memory 36, the several texts are received via the interface 38, and the method 10 according to one of the previous embodiments is executed.

[0032] The memory 36 can be a computer program product integrated into the device 32 or one that can be readily inserted into and / or removed from the device 32, such as a hard drive, an SSD, a USB stick (thumb drive), an optical data carrier, or another computer-readable storage medium. The device can be a computer.

[0033] Although the invention has been further illustrated and explained in detail by means of preferred embodiments, the invention is not limited by the disclosed examples, and other variations can be derived from them by a person skilled in the art without departing from the scope of protection of the invention. It is therefore clear that a multitude of possible variations exist. It is also clear that the embodiments mentioned as examples are truly only examples and are not to be understood in any way as limiting, for example, the scope of protection, the possible applications, or the configuration of the invention.Rather, the preceding description and the description of the figures enable the person skilled in the art to implement the exemplary embodiments in concrete terms, whereby the person skilled in the art, with knowledge of the disclosed inventive concept, can make various changes, for example with regard to the function or the arrangement of individual elements mentioned in an exemplary embodiment, without leaving the scope of protection defined by the claims and their legal equivalents, such as further explanations in the description.

Claims

[1] Artificial intelligence (AI)-based method (10) for detecting content contradictions between multiple texts, in which - the texts are each converted into a semantic vector (12), - at least one property of each of the vectors is determined (14), - Vectors whose properties are more similar to each other than the properties of at least one other vector are assigned to a group of properties (16), and - for texts that have been transferred to vectors assigned to the property group, an AI-supported check is carried out to detect content inconsistencies (18). [2] The method (10) according to claim 1, characterized by , that a property center of the property group is determined (20), wherein Vectors whose properties have a maximum predetermined distance to the property center are selected (22), and for texts whose vectors have been selected, AI-assisted testing for the detection of inconsistencies is carried out (24). [3] The method (10) according to claim 1 or 2, characterized by , that the vectors are assigned to different property groups (16), wherein The AI-assisted test for detecting contradictions between texts that have been converted into vectors that are assigned to one of the property groups is carried out separately from the AI-assisted test for detecting contradictions between texts whose vectors are assigned to another of the property groups (30). [4] The method (10) according to claim 1, characterized by , that the vectors are assigned to different property groups (16), wherein A property center is determined for each of the property groups. The AI-supported test for detecting contradictions between texts that have been converted into vectors that are assigned to one of the property groups, together with texts whose vectors are assigned to another of the property groups, is carried out when the distance between the property centers of two property groups is less than a predetermined threshold (30). [5] Device (32) for detecting content contradictions between several texts, with a processor (34), a memory containing instructions (36) which is connected to the processor (34) for data transmission, and an interface (38) connected to the processor (34) for data transmission and receiving multiple texts, characterized by , that the processor (34), when executing the instructions contained in the memory (36), receives the multiple texts via the interface (38) and then executes the method (10) according to one of claims 1 to 4. [6] Computer program product comprising instructions which, when the program is executed by a computer having a processor (34) and a memory containing multiple texts, cause the computer to execute the method (10) according to any one of claims 1 to 4. [7] Computer-readable storage medium comprising instructions which, when executed by a computer, cause it to execute the method (10) according to any one of claims 1 to 4.

Images