292 results about "Document Identifier" patented technology

A method and system for performing end-to-end encrypted semantic search and retrieval are disclosed. The system is configured to perform semantic searches of encrypted data, using product quantization and distance scores to facilitate efficient and secure retrieval of relevant data. User queries, indexes, document identifiers and document contents remain encrypted throughout the search process, offering a significant enhancement in privacy and security compared to known search methods. The method and system are particularly beneficial for applications that rely on semantic understanding in search queries, such as Generative AI and Retrieval-Augmented Generation (RAG), while maintaining strict data confidentiality.

A method may receive a query. A method may obtain a representation of the query, the representation approximating content of the query. A method may determine a set of files accessible by an operating system that are relevant to the query by comparing the representation of the query with representations of files stored in an index, the files in the set of files having representations meeting a similarity threshold with the representation of the query. A method may cause a display of a response to the query, the response identifying the files in the set of files as relevant to the query.

The invention relates to a tree neighbor context-based big language model RAG optimization method, which comprises the following steps of: reading a directory structure and content of a document, generating a corresponding directory tree, converting the directory tree into a Markdown format containing a title and a level, and automatically extracting according to a document template if no title exists; segmenting the document into knowledge blocks according to the priority; after the knowledge blocks are recalled, the context is expanded by using a tree adjacent context method, and a new knowledge block list is formed; grouping and sorting the expanded knowledge blocks according to document identifiers, splicing reference data based on different fields and confidence coefficients, and generating a character string set from the original document; and splicing prompt words by using the constructed reference data and the new knowledge block list, and sending the spliced prompt words to the large model for processing. According to the scheme, the performance and practicability of the large model in the information retrieval and answer generation process are comprehensively enhanced by optimizing context information, emphasizing key content, adapting to various document formats, improving user experience and the like.

The invention discloses an intelligent data information rapid checking and question-answering system, which relates to the technical field of intelligent data information and comprises a document import module, a vectorization analysis module, an NLP processing module, a question-answering matching module and a result output module. The document import module is used for receiving and importing documents including but not limited to PDF, DOC, TXT and PPT, analyzing the documents in different formats by adopting corresponding libraries and tools, identifying and classifying the files by identifying title formats and question number formats when processing the files in test questions and test paper formats, and performing preliminary analysis on the contents of the documents by utilizing a machine learning algorithm; according to the invention, the system not only can realize the rapid checking and question-answering functions of documents, but also can be applied to the field of educational training, realizes the online editing, management and test paper formation of test questions, reduces the manpower consumption in basic training business, and integrally improves the working quality and efficiency; meanwhile, the system can help enterprises to carry out internal training and examination, and improves the practicability and application range of the system.

The invention provides a file conversion method and device, equipment and a program product, and the method comprises the steps: obtaining a to-be-converted file which is an image file or a portable document format file, i.e., a PDF file; identifying elements in the to-be-converted file, determining attributes of the identified elements based on an identification result, and performing logic layering on the identified elements; the elements comprise texts; generating a structured document based on the logical layering result and the attributes of the identification elements; and converting the structured document or the edited structured document into an editable file in a portable document format, so that the PDF can be edited. According to the method and the device, the automatic conversion from the image or the PDF to the editable PDF is realized, and in the conversion process, through the structured document generated in the middle, while the logic level of the document is improved, a user is supported to carry out instant editing, and the editing efficiency and convenience are improved.

A method and system for performing end-to-end encrypted semantic search and retrieval are disclosed. The system is configured to perform semantic searches of encrypted data, using product quantization and distance scores to facilitate efficient and secure retrieval of relevant data. User queries, indexes, document identifiers and document contents remain encrypted throughout the search process, offering a significant enhancement in privacy and security compared to known search methods. The method and system are particularly beneficial for applications that rely on semantic understanding in search queries, such as Generative AI and Retrieval-Augmented Generation (RAG), while maintaining strict data confidentiality.

The invention discloses an unstructured file identification method based on primitive identification, and the method comprises the following steps: (1) preprocessing: cutting and zooming an electrical wiring drawing to a required size, and carrying out the gray processing; (2) primitive recognition: constructing a feature extraction network of an electrical primitive detection model by adopting a YOLO electrical primitive detection algorithm fused with an attention mechanism; (3) character recognition, which comprises the following steps: removing primitives, detecting an annotated text area, constructing a text candidate box, segmenting and adjusting an annotated text boundary, recognizing annotated text content, and filtering an electrical text recognition result; and (4) electrical element association relationship analysis, which comprises the following steps: frame element extraction, primitive group region division, template matching and electrical primitive-annotation text association. According to the method, the primitives and characters of the electrical wiring diagram can be obtained, the problem that different fonts and styles appear in marked characters can be solved, and semantic information can be understood.

In some aspects, the techniques described herein relate to a method including: receiving, at a query platform, a query and a document identifier; determining, based on the document identifier, a document-type parameter; generating, based on metadata related to the document-type parameter, an optimized query; generating a query vector embedding from the optimized query; executing a lexical search on a document pool, wherein the lexical search is based on the optimized query, and wherein the lexical search returns a first document set; executing a contextual search on a vector database, wherein the contextual search is based on the query vector embedding and returns a second document set; and generating a targeted document set from the first document set and the second document set.

The invention relates to the technical field of electric digital data processing, and provides a secondary development code cross-product transplantation method and device, computer equipment and a medium. The method comprises the steps of loading a source project, screening a to-be-transplanted source code file, identifying an original API type, determining a mapping relation between the original API type and a target API type, and generating an adaptation layer code framework according to the mapping relation; copying the to-be-transplanted source code file into the target project; on the premise of keeping the project hierarchical structure of the source project, replacing the calling node of the original API type with the calling node of the adapter type, and replacing the namespace reference with the corresponding namespace in the target project to obtain a migrated source code file; and obtaining a secondary development project of the target project based on the second product API according to the adaptation layer code framework and the migrated source code file. According to the method and the device, the transplantation efficiency and accuracy of the transplantation of the cross-product secondary development codes are improved.

A system for extracting a number of data elements from one or more data sources. The system may separate the text from the tables in a document, such that only the table data may be sent to the large language model (LLM), when the LLM only needs to review the table data. The system may include converting a PDF to text, and separating the tables form the document text using markdown language from converting the PDF. The system may form table chunks and text chunks, index the chunks using a vector embedding and store a chunk identifier, document identifier, and or a page identifier with the chunk to provide result traceability. The system may, in response to a prompt, retrieve and send the targeted table chunks or text chunks to the LLM to extract the data elements. The system populates an ontological data store based on the LLM response.

The invention relates to the field of information security, and particularly discloses a method for tracing confidential file leakage by using big data, which comprises the following steps of: respectively extracting file contents from suspected leakage files and original confidential files by using a file identification technology; and performing context semantic analysis on the suspected leaked file content and the original confidential file content based on deep learning so as to extract semantic features of the suspected leaked file content and the original confidential file content, and further performing fine-grained semantic comparison analysis on the suspected leaked file content and the original confidential file content based on a bidirectional attention mechanism so as to obtain the content of the suspected leaked file. And the deep semantic similarity and difference between the two files are revealed so as to judge whether the suspected leaked file is the variant of the original confidential file. Therefore, by performing fine-grained comparative analysis on the context semantic level on the suspected leaked file and the original confidential file, the association between the leaked file and the original confidential file after various processing can be more accurately captured, and the accuracy and reliability of traceability of confidential file leakage are improved.

In some examples, a code file in a development system is selected and the selected code file is segmented, using a language-specific parser, to create multiple parsed blocks. A block of the multiple parsed blocks is selected and a cryptographic hash function used to create a block signature that uniquely identifies the selected block. The block signature is compared to multiple signatures of blocks in third-party libraries. The multiple signatures are stored in an index. When the block signature matches one of the multiple signatures of blocks in third-party libraries, the index is used to determine a source file from which the selected block was copied. Vulnerabilities included in the source file and fixes to address the vulnerabilities are identified to enable a software developer to apply the fixes to the identified vulnerabilities.

The invention relates to a chemical enterprise knowledge base construction method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a document data set of a chemical enterprise, and sorting the document data set into independent knowledge units; and mapping each piece of document data to a high-dimensional vector space through an embedding mechanism of a large language model to generate an embedded vector, and storing the embedded vector and the corresponding document identifier into a vector database. Calling an embedded function to convert a natural language query text input by a user into a high-dimensional query vector, and calculating a similarity score between the high-dimensional query vector and each embedded vector in the vector database through vector similarity measurement so as to select associated document data with the similarity score exceeding a first threshold value. And splicing the natural language query text and the context of the associated document data into the input of an RAG generation model, generating an answer through a large language model, and constructing a knowledge base based on the generated answer in combination with the associated document data.

The invention discloses a graphical file editing system and method. The system is deployed in the form of an IDE plug-in, and comprises an editor extension module of a JBCefBrowser embedded browser, an editor extension module of a JBCefBrowser embedded browser, and an editor extension module of a JBCefBrowser embedded browser, the file identification module is used for scheduling a corresponding editor based on a file suffix; the graphical engine is used for performing classified rendering on the JSON / XML data based on the component metadata registry; the event processing module supports three-stage dragging and eight-direction zooming; an event driving communication module based on window.post Message, and an event driving communication module based on window.post Message; the operation module is used for synchronizing files and codes through a PSI interface; and a file template management module based on a. Ft template. The method comprises the steps of editor registration and embedding, file identification and loading, content analysis and rendering, interaction event processing, data persistence synchronization and template file creation. According to the method, graphical dragging and editing of the structured file are realized in the IDE, and the development efficiency and the normalization are remarkably improved.

The invention relates to the technical field of information retrieval, and provides a dialogue type generation retrieval method and system based on large model document index perception. The dialogue type generation retrieval method based on large model document index perception comprises the steps that according to user query input, a trained large language model is adopted, and an information sorting list related to the user query input is output; on the basis of a corpus, a cross attention layer is adopted, key information of a context is extracted, a proposition is obtained, and a document identifier is obtained; in the first-stage training, generating loss is introduced to generate information related to the current query, and a document identifier is generated; in the second-stage training, a ranking list of document identifiers retrieved by comprehensive loss optimization is introduced; and outputting a paragraph ranking list corresponding to the document identifier through a beam search decoding strategy. Through innovative document identifier design and training strategies, more effective context understanding and denoising are realized, and the dialogue type retrieval accuracy and user experience are improved.

One or more embodiments of the invention provide a training method of a generative retrieval model, and an information retrieval method and device. The training method adopts a two-stage training strategy. Wherein the first stage can be called a document identifier generation stage, and in the stage, first query entries and first standard document identifiers which are in one-to-one correspondence are utilized to train a to-be-trained generative retrieval model to learn the relationship between query and document identifiers. The second stage can be called a document ranking stage, and in the stage, the relationship between the model trained in the first stage is trained by utilizing second query entries, second positive sample documents and negative sample documents which are in one-to-one correspondence through a comparative learning method to learn the query and the documents.

The application relates to the technical field of text label marking, and provides a text specification marking method and system based on a rule corpus, which comprises the following steps: analyzing a policy and regulation document, identifying and marking condition morphemes and conclusion morphemes in the policy and regulation document, constructing a logical relationship between the two by using a large language model, and forming a rule corpus composed of structured morpheme pairs; performing semantic embedding on the corpus to generate a semantic vector library; performing multi-label coding on a verification data set based on the rule corpus, and constructing a multi-label training data set; training a deep learning classification model by taking semantic vectors as features and multi-labels as targets, so that a text specification marking model is obtained; and automatically marking target text by using the model. The application significantly improves the accuracy, interpretability and business adaptability of text marking, improves the update quality of a system label data set, and reduces the system maintenance cost.

The invention relates to the technical field of artificial intelligence, in particular to a multi-modal intelligent compliance pre-auditing method and system, and the method comprises the following steps: receiving a qualification package uploaded by a supplier, and automatically completing the decompression operation; accurately marking the decompressed page as five types of a subject license, a financial report, industry qualification, a product catalog and environmental protection and safety evaluation by using a layout perception model, and executing fingerprint hash de-duplication processing on the homologous file to generate a unique document identifier DocID; the method has the beneficial effects that the authenticity verification, financial analysis and directory matching of all qualifications of a single supplier can be completed within about 30 seconds by adopting a one-package type automatic sorting link and parallel API aggregation verification; and compared with the traditional manual item-by-item check which merges the process of several hours, the whole check period is shortened by more than 90%, and the manpower of compliance specialists is greatly released.

The invention provides a data access method and related equipment, the method is applied to an object storage service node, the object storage service node comprises a hierarchical namespace, the hierarchical namespace is used for mapping at least one object into a file or a directory on a directory tree, and the method comprises the following steps: receiving a file creation request, the file creation request carries a first file name and a first file identification code of a parent directory of a first file; obtaining metadata of a parent directory of the first file corresponding to the first file identification code; according to metadata of a parent directory of the first file, the first file is created under the parent directory, a second file identification code is distributed to the first file, and the first file is a file or a directory on a directory tree; and recording a corresponding relation between the second file identification code and the first file name into a file index table. Therefore, the user uses the file identification code to access the file or the catalog stored by the object storage service node, the file full path does not need to be analyzed layer by layer, and the access efficiency of the user is improved.

The invention discloses a bidding file data processing method and system, and the system employs a MoE mixed expert model architecture, and comprises a multi-type bidding file recognition module, a routing network, a professional sub-model library, a bidding file analysis engine module, and an automatic bidding file generation module. Wherein the bid invitation file is input into the multi-type bid invitation file identification module so as to identify the type of the bid invitation file; according to the type of the bid invitation file, calling a corresponding sub-model in a professional sub-model library; then the bid invitation file analysis engine module analyzes the bid invitation file; and the automatic bidding document generation module generates a bidding bidding document.

The invention discloses an offline upgrading method and device of a food processor and electronic equipment, and relates to the field of data processing. The method comprises the following steps: detecting whether pluggable storage equipment connection exists or not; if it is determined that the pluggable storage device is connected, triggering a device identification program; according to the device identification program, enumerating each file under a root directory and a subdirectory of the pluggable storage device; reading file header information of each file, and extracting a file type identifier of each file according to the file header information; judging whether the file type identifier is matched with a preset type; if it is determined that the file type identifier is matched with the preset type, identifying a file corresponding to the file type identifier as a target upgrade file; and based on the target upgrading file, performing incremental updating on a system file of the food processor. By implementing the technical scheme provided by the invention, system upgrading and data management can be carried out in an offline state, and the user experience is improved.

The present application relates to the technical field of parallel processing, and provides a parallel processing method and system for batch write-off tasks. The method comprises: extracting a plurality of pieces of query data of a preset write-off window, comprising external write-off codes and write-off services; performing file import on the basis of the data, and generating transaction files with unique identifiers; when file name identifiers are verified, parsing the external write-off codes, and generating fund files; generating pending write-off files by means of procurement-settlement mapping authentication; identifying request time identifiers to identify write-off conflicts, and generating conflicting file clusters; updating the configuration of a parallel processing module on the basis of the conflict condition; and utilizing the updated module to process the conflicting file clusters in parallel. The present application solves the technical problems of data conflict, irrational resource allocation and low processing efficiency caused by improper parallel processing in batch write-off tasks, and realizes real-time extraction, verification, parsing and authentication of write-off data, thereby ensuring the data accuracy and consistency.

The invention relates to the technical field of semantic understanding, in particular to a steel structure archive retrieval system based on semantic understanding, which comprises a semantic index construction module for reading a document text sentence by sentence based on an original document of a steel structure archive, matching a preset sensitive information ontology dictionary and establishing a steel structure archive security index. According to the method, the original document text of the steel structure archive is read sentence by sentence and matched with the preset sensitive information ontology dictionary to realize the identification of the special node construction and the sensitive information in the document, so that the document content can complete the effective labeling of the safe sensitive information in the index stage, and the safe grading of the archive data is realized from the source. According to the method, document identifiers and sensitive concept categories are aggregated in combination with role identifiers and retrieval keywords in user retrieval requests, document lists meeting conditions are screened out, document access permissions are compared with preset rules item by item, a content rewriting trigger instruction is formed through combination, and it is effectively ensured that user query results are strictly limited by the permission range.

The embodiment of the invention discloses a data retrieval method and device.The method can comprise the steps that a search condition is received, a first structured object is generated, and the first structured object comprises a query statement contained in the search condition; based on the first structured object, a score operator is created, the score operator comprises the first structured object, M document identifiers matched with the query statement and a scoring component, and M is a positive integer larger than 0; m first documents matched with the M document identifiers are obtained, and score description information corresponding to each first document in the M first documents is generated through a score component; based on the score description information corresponding to each first document and the first structured object, M second structured objects are generated, and one second structured object corresponds to one first document. By adopting the embodiment of the invention, the problem of time delay caused by acquiring detailed information of the search result during data retrieval can be solved, the method and the device are suitable for more business scenes, and the retrieval efficiency is improved.

The invention provides an AI-based electric power engineering archive knowledge service method and system in the technical field of electric power engineering archive management. The method comprises the following steps: S1, obtaining a historical archive text to construct a data set; s2, training the archive knowledge extraction model based on the data set; mining an archiving rule from historical archiving information collected from the data set through a big data technology; s3, acquiring a to-be-copied electric power engineering archive file, and performing copy identification; s4, identifying an electric power project file text from the electric power project file; s5, inputting the electric power engineering file text into the file knowledge extraction model to obtain file knowledge so as to construct a file knowledge graph; s6, extracting archiving information from the electric power engineering archive file, generating an archiving reminding notification based on the archiving information and an archiving rule, and executing archiving operation; and S7, recording the archiving characteristics in the archiving process to carry out compliance check. The method has the advantage that the quality and efficiency of electric power engineering file management are greatly improved.

The present invention relates to the field of document recognition technology, and discloses a method and system for automatically identifying regulations associated with letters. The method comprises the steps of file uploading, preprocessing, text recognition, chapter recognition, content extraction, tree structure building, and result display. In the image preprocessing, the present invention has accurate grayscale, flexible noise reduction, adaptive binarization, and efficient tilt correction, thereby improving image quality and ensuring the basis for text recognition. The text recognition process is innovative, and line segmentation and character segmentation are reliable and accurate. Character recognition combined with a standard character template library is intelligent and efficient, thereby improving accuracy. Table of contents recognition is judged in multiple dimensions through regular expressions, font features, and numbering structures, and the hierarchy is scientifically and verified, and the content extraction is reasonable. The tree structure is built based on the hierarchy, combined with refined node and content pairs, with clear levels. The modules of the system work together to achieve automated processing, providing users with convenient and intuitive display of regulatory information.

The invention relates to the technical field of text label marking, and provides a text standard marking method and system based on a rule corpus, and the method comprises the steps: analyzing a policy and regulation file, recognizing and marking a condition morpheme and a conclusion morpheme in the policy and regulation file, constructing a logic relation between the condition morpheme and the conclusion morpheme by using a large language model, and marking the rule corpus; forming a rule corpus consisting of structured morpheme pairs; performing semantic embedding on the corpus to generate a semantic vector library; performing multi-label coding on the verification data set based on the rule corpus, and constructing a multi-label training data set; training a deep learning classification model by taking the semantic vector as a feature and taking multiple tags as a target to obtain a text specification marking model; and automatically marking the target text by using the model. According to the method, the accuracy, the interpretability and the service adaptability of text marking are remarkably improved, the updating quality of the system label data set is improved, and the system maintenance cost is reduced.

The invention discloses a symmetric searchable encryption method and device with a hidden search mode, equipment and a medium, and relates to the technical field of data retrieval, and the method comprises the following steps: obtaining an encrypted document uploaded by a data owner and a corresponding security index, respectively converting the security index into a corresponding first vector and a corresponding second vector through a bloom filter and a confusion bloom filter of the cloud server; when a keyword query vector sent by a data user is received, performing matching calculation on the first vector, the second vector and the keyword query vector based on a preset homomorphic encryption algorithm and the homomorphic encryption public key to obtain a corresponding matching calculation result; and sending a matching calculation result to the data user, so that the data user decrypts the matching calculation result by using the homomorphic encryption private key to obtain a corresponding first vector dot product result and a corresponding second vector dot product result, decrypting the first vector dot product result and the second vector dot product result by using the pseudo-random encryption shared key, and sending the decrypted result to the data user. And obtaining a corresponding plaintext document identifier set containing the keyword.

The invention provides an automatic security vulnerability detection method and system for multiple types of scripts in an embedded system, electronic equipment and a readable storage medium, the method comprises the following steps: unpacking and analyzing firmware of a to-be-detected embedded system to identify a script file and a template file; identifying a dependency relationship between script files and template files and interaction and dependency relationships between different types of script files, marking sensitive input source points and potential vulnerability sink points, and generating a control flow graph and an abstract syntax tree by adopting a context-sensitive control flow analysis algorithm; constructing a power set lattice according to an assignment statement in the control flow graph, and performing data flow analysis by using a fixed point algorithm to obtain a data flow constraint relationship; and performing taint analysis to identify all dangerous execution paths. The script security vulnerability can be automatically identified, the security of the embedded system in the script execution process is improved, and potential security threats are effectively prevented.

The invention discloses a rapid identification and grading early warning method for sensitive files of customs field mobile equipment, and relates to the technical field of electronic data forensics, and the method comprises the following specific steps: constructing a sensitive key feature library, collecting and analyzing mirror image data, fusing and judging sensitive information, and automatically generating a sensitive grading early warning and risk report. According to the method, sensitive information samples of customs law enforcement cases, supervision notification cases and compliance public data sources are collected, and accurate extraction of four modal features of images, texts, audios and videos is ensured by using a multi-modal sensitive feature extraction formula; in the preliminary screening stage, file feature vectors are compared in batches through a preliminary screening similarity calculation formula, and suspicious files are quickly marked; in the recognition stage, multi-dimensional matching is adopted for files of different formats, and recognition confidence coefficients of all modes are generated; finally, whether the to-be-detected file is a sensitive file or not is judged through a comprehensive confidence coefficient calculation formula, and the omission ratio is reduced.