47 results about "Document similarity" patented technology

The invention discloses an information management method and system based on an electronic bidding transaction platform, and relates to the technical field of information management. By constructing a multi-dimensional and multi-level information management system, deep mining and intelligent identification of potential association relationships and behavior patterns among bidding units are realized, so that the capability of the electronic bidding platform in the aspects of risk prevention and control and abnormal behavior identification is effectively improved. By fusing multi-source heterogeneous data such as industry and commerce registration information, bidding history, project contracts and the like and introducing atlas modeling and semantic analysis technologies, key risk factors such as legal person penetration, historical cooperation, high bidding document similarity and the like can be accurately identified, then the risk association strength between bidding units is quantitatively evaluated, and the risk assessment efficiency is improved. And a visual map and an auxiliary auditing mechanism are formed, so that the judgment efficiency of supervisors is improved.

The invention provides a document similarity comparison method and system based on deep semantics and shallow layout. The method comprises the following steps: S1, constructing a twin network model based on a twin network architecture; s2, training data are obtained, the training data comprise labeled document sample groups, and each document sample group comprises two training documents and similarity labeled values of the two training documents; s3, training the twin network model by using the training data, so that the twin network model can learn deep semantic features and shallow layout features of the training document at the same time; s4, performing feature extraction and similarity calculation on the two to-be-compared documents by using the trained twin network model to obtain the similarity of the to-be-compared documents; and S5, based on the calculated similarity of the to-be-compared documents, judging whether the two to-be-compared documents have a plagiarism relationship or not by using a dichotomy model. According to the method, the deep semantic features and the shallow layout features of the document are considered at the same time, so that the accuracy of similarity comparison can be remarkably improved.

The invention discloses a multi-document semantic similarity comparison system and method based on a knowledge vector library, and relates to the technical field of data comparison, and the system comprises a construction unit of the knowledge vector library, a document similarity comparison unit, a document preprocessing module, a key element recognition module, a multi-level abstract generation module and a knowledge vector library generation module. A vectorization comparison module, a large language model analysis module and a result comprehensive report module. The invention further provides a multi-document semantic similarity comparison method based on the knowledge vector library. According to the method, deep semantic understanding and accurate similarity analysis of the document are realized by combining a large language model, vectorization comparison and a multi-level abstract technology. Through automatic identification and directional comparison of key elements, the system can capture essential differences among documents, and provides high-quality and interpretable similarity analysis results for users.

A system may include a processor and a non-transitory computer readable medium having stored thereon instructions for performing operations including obtaining a first dataset including a plurality of electronic documents and a plurality of entities, extracting, based on one or more rules, one or more subgraphs including representations between one or more electronic documents and one or more entities from the first dataset, identifying one or more groups in the one or more subgraphs, each group including electronic documents and entities selectively identified from a corresponding subgraph based on the representations, and learning the representations associated with the electronic documents and the entities based on the one or more groups and updating the representations in the first dataset. The first dataset may include data corresponding to contextual information associated with the plurality of entities and the representations may be determined based on this contextual information.

The retrieval enhancement method for the threat intelligence knowledge graph provided by the invention comprises the following steps: preprocessing threat intelligence information to obtain block intelligence documents; performing entity extraction on the partitioned intelligence document to obtain an intelligence entity group; performing expansion classification and entity attribute information completion on the entities in the intelligence entity group to obtain completed intelligence entities, determining a relationship between the intelligence entities, and establishing a threat intelligence knowledge graph according to the relationship between the intelligence entities; performing entity matching and block intelligence document similarity matching according to question and answer statement input to obtain a related intelligence document block set; weighting, mixing and rearranging the partitioned intelligence documents in the related intelligence document block set to obtain a final related intelligence document block set, and generating an input answer corresponding to the question and answer statement according to the final related intelligence document block set. By applying the method, high-dimensional semantic hierarchy representation can be better carried out on heterogeneous data source intelligence, semantic information can be well understood and inquired, and therefore the quality of large model generation content is enhanced.

A computer-implemented method for document clusterization, comprising: receiving an input document; determining, by evaluating a document similarity function, a plurality of similarity measures, wherein each similarity measure of the plurality of similarity measures reflects a degree of similarity between the input document and a corresponding cluster of documents of a plurality of clusters of documents; based on the plurality of similarity measures, determining that the input document does not belong to any of the clusters of documents of the plurality of clusters of documents; creating a new cluster of documents; and associating the input document with the new cluster of documents.

The invention relates to a document retrieval method based on multi-field information and outlier detection. The method comprises the following steps: acquiring document query information for retrieving documents, and acquiring field similarity between the document query information and different field information of each candidate document; if at least one high outlier exists in the current field similarity between the document query information and any one piece of field information of the current candidate document, obtaining the document similarity between the document query information and the current candidate document according to each high outlier; if the high outlier does not exist in the current field similarity, obtaining the document similarity between the document query information and the current candidate document according to at least one current field similarity; and obtaining a document retrieval result corresponding to the document query information according to the document similarity between the document query information and each candidate document. By adopting the method, the correlation of document retrieval results can be improved.

Secure distributed document discovery via vector similarity is performed by a data discovery service. The service receives a request to perform a search using input data. The service sends the input data and an embedding protocol to different data repositories that comprise vector databases. The service receives a results from the different data repositories; each result identifies documents and corresponding document similarity scores. The service generates an overall ranking of the documents according to the document similarity scores. The service returns a query result that identifies the at least a portion of the documents and indicates the ranking of the identified documents.

The invention discloses a science and technology project declaration duplicate checking method and system based on a large language model and a Jaccard similarity coefficient, belongs to the technical field of natural language processing, and aims to solve the technical problem of how to improve the accuracy and efficiency of science and technology project declaration duplicate checking. According to the technical scheme, the method comprises the steps that document core content is extracted, wherein the core content of a science and technology project document to be subjected to duplicate checking is extracted through a large language model subjected to data training of a plurality of science and technology projects; splitting document segments: splitting the document core content into a plurality of document segments based on a natural language processing technology; document vectorization storage: converting document fragments into vectors through a text embedding model, and storing the vectors in a vector database; calculating a vector distance and retrieving historical items: calculating the Euclidean distance or cosine similarity between the vector of the item document fragment to be subjected to duplicate checking and the vector of the historical item document fragment, and extracting topK historical items with the closest distance; calculating a Jaccard similarity coefficient; aggregating the document similarity; and generating a duplicate checking result.

The invention discloses a multi-mode document retrieval method and device and electronic equipment. The method comprises the following steps: acquiring document retrieval demand parameters; determining a document retrieval vector corresponding to the document retrieval demand parameter; a target document set is called, the target document set comprises to-be-selected documents corresponding to multiple to-be-selected document vectors respectively, the multiple to-be-selected document vectors are determined according to corresponding document fusion features, and the corresponding document fusion features are obtained according to the corresponding multi-modal data; respectively determining document similarities between the document retrieval vector and the plurality of document vectors to be selected; and according to the target document set and the plurality of document similarities, determining a target document corresponding to the document retrieval requirement parameter from the plurality of to-be-selected documents. According to the method and the device, the technical problem that a retrieval result is inaccurate when a multi-modal document is retrieved in related technologies due to complexity of multi-modal data is solved.

The application discloses a text classification method based on document similarity, and belongs to the technical field of document processing. The method updates the word set of a document sentence through iterative optimization of a word segmentation mechanism and generates a first sentence vector, and generates a first similarity vector in combination with a sentence weight and the first sentence vector; a plurality of theme words are extracted from the word set according to an entity weight and a theme feature vector is generated, a knowledge supplement vector is extracted from a knowledge base based on the context vector of each theme word and the entity weight, a theme enhancement vector is generated in combination with the theme feature vector and the knowledge supplement vector, and a second similarity vector is generated based on the theme enhancement vector; after the first similarity vector and the second similarity vector are compared and optimized in terms of difference, a document similarity vector is obtained and a text category is output. Through iterative word segmentation optimization, theme-guided knowledge enhancement and a similarity comparison mechanism, the application can effectively improve the reliability and accuracy of text classification.

The application relates to a method and device for identifying official documents, computer equipment and a storage medium. The method comprises the following steps: obtaining a to-be-identified file; extracting text information in the to-be-identified file; matching the text information with reference official document elements in a reference official document element set to obtain a first official document element set, wherein the first official document element set is composed of first official document elements matched successfully with the reference official document elements in the text information; calculating a target confidence degree corresponding to the to-be-identified file according to a first confidence degree corresponding to the first official document elements in the first official document element set; and determining that the to-be-identified file is an official document if the target confidence degree meets an official document similarity condition. The method can realize automatic identification of official documents and improve the identification efficiency of official documents.

The invention discloses a knowledge management method and system based on document similarity detection and a multi-modal knowledge base. The method comprises the steps of obtaining a target document; when a target document is put in storage, a similarity detection method based on a resource self-adaption mechanism is adopted, and existing documents are screened out from the multi-modal knowledge base to serve as a candidate document set; a similarity detection method based on a precision and efficiency optimal mechanism is adopted, and the candidate document set is used for conducting duplicate removal on the target document or storing the target document which is judged to be a new document; obtaining a question asked by the user; based on the multi-mode knowledge base subjected to duplicate removal or updating, multi-path mixed retrieval is conducted on the problem, a multi-path mixed retrieval result is subjected to two-stage sorting to obtain a rearrangement result, reference data are selected from the rearrangement result, the problem, cue words, context and the reference data are assembled into a large model input instruction according to a preset cue word template, and the large model input instruction is input into the multi-mode knowledge base. An answer generated by the large model is obtained and returned to the user; and comprehensive improvement of the knowledge utilization rate, the retrieval precision and the efficiency is realized.

One embodiment provides a method for identifying similar objects by performing document attribute comparisons, the method including: providing a digital standard system that includes a user interface and a data store of digital standards; receiving a request for a similarity comparison; performing the similarity comparison; generating a document similarity score for each of the digital standards within a group of digital standards; and displaying at least one of the digital standards from the group based upon the document similarity score. Other aspects are described and claimed.

The present invention discloses a method for determining document similarity based on image, video, and text content simultaneously, comprising the following steps: S1: selecting two documents D1 and D2 of the same type, performing a hash calculation on them, and determining whether D1 and D2 are duplicate documents; if D1 and D2 are not duplicate documents, performing text, image, and video content similarity calculations on documents D1 and D2, respectively, setting weights for the text, image, and video similarities of the documents, obtaining document similarity through weighted calculation, and comparing the document similarity with a preset threshold to draw a conclusion on document similarity. The present invention determines both text similarity and image and video similarity, calculates document similarity after comprehensive analysis, and prompts a human to perform a recheck for subsequent processing.

The application discloses a kind of wisdom legal inquiry method based on multi-wheel pruning Skyline algorithm, comprising: local equipment of legal department obtains local legal case according to the query command issued by central server;The local legal case obtained is uploaded to the central server in encrypted form, the central server carries out integration analysis, and the legal case with higher comprehensive document similarity is sent to local equipment.The scheduling strategy module of the central server obtains the physical node of local legal department or the corresponding legal department priority information according to the attributes such as name, gender, native place, address, case name, case-cracking event, capture event and report event in legal case.The application can make legal information retrieval personnel free from reading a large number of cases, save a lot of time and manpower and material resources, especially provides technical guarantee for finding similar cases, case-cracking clues and inducing crime trend.

The present application provides a generative multi-document summarization method using Entity explicit graph to solve the problems of the prior art. The Entity graph based on SimBert is used instead of the traditional document similarity graph to express the connection between documents. Entity extraction is performed on each paragraph, and the Entity of each paragraph is spliced into a sentence. Then, the cosine similarity between all paragraphs is calculated using SimBert. After threshold filtering, an explicit graph representing the connection between paragraphs is obtained. The graphing method based on neural networks and aimed at the entities within the document is more effective than the inflexible rules defined by humans. In addition, the present application improves the attention mechanism and hierarchical graph attention mechanism of the graph perception, introduces a gating mechanism and a residual connection, so that the explicit graph and implicit graph information can be better integrated, and the implicit relationship learned by the attention mechanism is reserved with a residual path to ensure that the information learned by the network occupies a more important position, thereby guiding the generative multi-document summarization.

This application provides a document similarity determination method, a computer-readable storage medium, an electronic device, and a computer program product. The method includes: determining the semantic similarity between a first document and a second document; determining the structural similarity between the first document and the second document; determining the weights of the semantic similarity and the structural similarity; and determining the similarity between the first document and the second document based on the semantic similarity, the weights of the semantic similarity, the structural similarity, and the weights of the structural similarity. Therefore, it can at least solve the problem in related technologies where document similarity determined based on extracted semantic information is not accurate enough, enabling a more comprehensive assessment of the similarity between documents and improving the accuracy of similarity determination.

This invention discloses a text classification method based on document similarity, belonging to the field of document processing technology. The method iteratively optimizes the word segmentation mechanism to update the word set of document sentences and generates a first sentence vector. A first similarity vector is generated by combining sentence weights and the first sentence vector. Multiple topic words are extracted from the word set based on entity weights to generate topic feature vectors. Knowledge supplement vectors are extracted from a knowledge base based on the context vectors of each topic word and entity weights. A topic enhancement vector is generated by combining the topic feature vector and the knowledge supplement vector. A second similarity vector is generated based on the topic enhancement vector. The first and second similarity vectors are then optimized through difference comparison to obtain the document similarity vector, and the text category is output. This invention, through iterative word segmentation optimization, topic-guided knowledge enhancement, and a similarity comparison mechanism, can effectively improve the reliability and accuracy of text classification.

A computer-implemented method for document clusterization, comprising: determining, by evaluating a first document similarity function, a first plurality of similarity measures, each similarity measure of the first plurality of similarity measures reflecting a degree of similarity between the input document and a corresponding cluster of documents of a plurality of clusters of documents; based on the plurality of similarity measures, determining that the input document belongs to a subset of the plurality of clusters of documents; determining, by evaluating a second document similarity function, a second plurality of similarity measures, wherein each similarity measure of the second plurality of similarity measures reflects a degree of similarity between the input document and a corresponding cluster of documents of the subset of the plurality of clusters of documents; associating the input document with a cluster of documents associated with a maximum similarity measure of the second plurality of similarity measures.

The invention relates to the technical field of data processing and information security, in particular to a document similarity detection method based on multi-modal content fusion, which comprises the following steps: a multi-modal feature extraction step: analyzing a target document into text, image and video data, extracting deterministic hash fingerprints and constructing semantic feature vectors; a difference entropy value calculation step: calculating the Hash similarity, calculating the semantic similarity when the Hash similarity is lower than a threshold value, and mapping the difference between the Hash similarity and the semantic similarity into a modal decision confidence entropy; a game strategy calculation step: collecting system calculation resource load data, and calculating a cost perception factor by using a dynamic game strategy model in combination with confidence entropy; a dynamic fusion and survival step: distributing an asymmetric fusion weight based on the perception factor, and generating a detection result; when the load exceeds the limit and the confidence entropy indicates a high risk, triggering a degradation survival mechanism; according to the invention, the recall ratio and throughput can be automatically balanced in a resource-limited scene, and system avalanche caused by single-point attack is prevented.

The present invention relates to the technical field of natural language processing, and discloses a lightweight chemical literature similarity calculation method. The model of the present invention includes a multi-head self-attention mechanism feature extraction module, and further extracts the features of text through a convolution layer and a pooling layer. The specific steps of the method for calculating the similarity of chemical literature are: using a crawler to perform data mining and preprocessing of chemical literature, using a self-attention mechanism and residual normalization and convolution and pooling ideas to build a backbone network, using the DTW function to compare the similarity of two documents, and finally pruning the network model to make the model structure efficient and simple, which is more conducive to the migration of the model. Compared with the prior art, the present invention solves the problems of non-standard search terms and semantic ambiguity in document review, and can quickly calculate the similarity of texts.

The present invention relates to a method and a readable storage medium for calculating the similarity of a chapter-level document. The method comprises: obtaining m first sentences of an original document and n second sentences of a comparison document; performing Chinese word segmentation and stop word removal processing on the first sentence and the second sentence to obtain a general dictionary; obtaining a common annotation based on the word frequency and part of speech of the remaining words after the stop words are removed; obtaining the word frequency and inverse document frequency value of the word based on the common annotation of the word to obtain the word frequency and inverse document frequency vector of the first sentence and the second sentence; finding the second sentence most similar to each first sentence of the original document from the comparison document to obtain a first similarity score; obtaining a final document vector output by the final Doc2Vec; obtaining a second similarity score based on the final document vector; and determining a similarity result based on the average of the first and second similarity scores. The method takes into account word frequency, part of speech and semantics, and can effectively process long texts.

The invention discloses an electric power science and technology project document duplicate checking method based on a knowledge graph. The method comprises the steps that an existing electric power science and technology project document is acquired and preprocessed; performing knowledge extraction to construct a dictionary and a knowledge graph of the electric power science and technology project document; fusing the dictionary of the electric power science and technology project documents with the universal language lexicon to obtain a professional lexicon of the electric power science and technology project documents; performing word segmentation vectorization processing on the target electric power science and technology project document; calculating the document similarity of the target electric power science and technology project document; and duplicate checking of the target electric power science and technology project document is completed according to the document similarity. The invention also provides a system for realizing the power science and technology project document duplicate checking method based on the knowledge graph, and a storage medium comprising the power science and technology project document duplicate checking method based on the knowledge graph. According to the method, duplicate checking of the electric power science and technology project document is achieved, the reliability is higher, the accuracy is better, and the efficiency is higher.

The present invention relates to a legal and regulatory retrieval system and method based on semantic information weighting, belonging to the field of information. The system comprises an index generation module, a domain analysis module, a semantic weight assignment module, and a score normalization module. The index generation module is used to segment and index the data in the database by introducing an IK Analyzer, generate an index information table, and merge the data to form an index database; the domain analysis module is used to input the query statement input by the user into a trained domain analysis model for domain analysis, and match the domain analysis results with the index database to obtain domain-related temporary index information; the semantic weight assignment module is used to assign weights to each term in the query statement, and incorporate the weights assigned to each term into the document similarity calculation to comprehensively obtain the document score; and the score normalization module is used to normalize the document score. The present invention effectively improves the retrieval accuracy and optimizes the retrieval results.

To provide a similar document search device and a program which allow search of a document which fits a context.SOLUTION: A similar document search server 1 includes: an item-set storage unit 23 for storing, in association with stored documents, a plurality of item sets each being a combination of one item name and one item value corresponding thereto; an item-set extraction unit 14 for extracting a plurality of item sets from a target document image; and a similarity calculation unit 15 for calculating document similarity by using the plurality of item sets extracted by the item-set extraction unit 14 and the plurality of item sets corresponding to the plurality of stored documents stored in the item-set storage unit 23.SELECTED DRAWING: Figure 1

Systems and methods for assessing similarity of documents are provided. Embodiments of the systems and methods include extracting a reference document text from a reference document, extracting an archived document text from an archived document, and quantifying the reference document and the archived document. The systems and methods may also include determining a document similarity value of the quantified reference document and the archived document. Determining the document similarity value includes calculating a set of vector similarity values for a set of combinations of a reference document text vector and an archived document text vector, and calculating the document similarity value, including a sum of the plurality of vector similarity values.

This application relates to the field of artificial intelligence, and provides a document search method, apparatus, electronic device, and storage medium based on artificial intelligence. The document search method based on artificial intelligence includes: obtaining a first document data set; constructing a document vector matrix based on the first document data set; obtaining a second document data set based on the document vector matrix, where the second document data set contains stop words; matching the stop words in the second document data set according to a preset stop word list to construct a document matching vector and obtain a document similarity value; and filtering the second document data set based on the document similarity value to obtain a filtered document. This application can effectively reduce the computational complexity in the stop word filtering stage during document search and improve the matching efficiency of document search.

The invention relates to the technical field of artificial intelligence, and discloses an automatic document duplicate checking and rewriting method and device and a program product, and the method comprises the steps: obtaining a target document, carrying out duplicate checking comparison on the target document according to a reference document, generating a corresponding document similarity, and carrying out the matching of terms in the target document according to a user-defined term library. Proprietary terms in the target document are determined, other contents except the proprietary terms in the target document are rewritten according to the language model, and a rewritten document is obtained. According to the method, the corresponding document similarity is generated through automatic duplicate checking comparison, the special terms in the document are protected, then other contents except the special terms in the target document are rewritten according to the language model, and the two steps of duplicate checking and rewriting are connected, so that the number of times of repeatedly switching tools by a user is reduced, the labor cost and the knowledge cost are reduced, and the user experience is improved. The process of duplicate checking and rewriting is shortened, and the efficiency of duplicate checking and rewriting is improved.