4709 results about "Directory" patented technology

A method, apparatus, and computer-readable medium of a migration platform for analyzing a legacy database based on applying a large language model to the legacy database and legacy application code to extract legacy database values and legacy application values, determining destination database values corresponding to a destination database based on the legacy database values and the legacy application values, generating migration code for migrating the legacy database to the destination database based on code generation templates, user preferences, and code generation procedures configured to create one or more files and one or more directories based on the code generation templates, validating the migration code based on generating mock data, storing the mock data in the legacy database, and applying the migration code to the mock data to migrate the mock data to the destination database, and executing the validated migration code to perform a migration of the legacy database.

The invention provides a multi-modal intelligent question-answering system based on a large model and a construction method and device, and the method comprises the steps: carrying out the region recognition of a PDF document, and obtaining a table region picture, a text region picture and / or an image region picture; the Word document is analyzed, paragraph content of the Word document is obtained, and the paragraph content is composed of the first text, table data and / or Word pictures; constructing a knowledge base; wherein the knowledge base comprises vector representation of document slices; the document slice is determined based on the table area picture, the character area picture and / or the image area picture, or the document slice is determined based on the paragraph content and the corresponding chapter information thereof. Visibly, according to the method, layout recognition can be carried out on the PDF document, catalog chapter recognition can be carried out on the Word document, the knowledge base is constructed based on the layout recognition, the semantic integrity and coherence of multi-modal text question and answer recall can be improved, and the answer accuracy is improved.

The invention provides an integrated metadata management method and system based on data source expandability, and relates to the technical field of data sources, and the method comprises the steps: executing real-time detection on a plurality of heterogeneous data sources through a PF4J plug-in framework, collecting connection information, and establishing a dynamic connection pool; obtaining a table structure, a field attribute and an association relationship list, and assembling the table structure, the field attribute and the association relationship list into JSON format data; performing analysis and standardization processing on the data, and constructing an association degree calculation matrix; generating a standardized data structure description file; establishing a three-layer index directory, and detecting structure change and updating; and calculating a data table reference dependency relationship, and generating a DAG data flow path. According to the method, efficient integration and management of heterogeneous data sources are realized, and the metadata processing efficiency is improved.

The invention belongs to the technical field of data storage, and discloses a multi-source marine environment data storage method and system based on HDF5. According to the method, a hierarchical storage framework is constructed, a domain-type-source three-level directory architecture is constructed, a space-time dimension dynamic extension mechanism is executed, and space-time composite coding is performed; designing an HDF5 nested packet storage architecture, wherein grid topological structure definition, dynamic resolution self-adaptive storage, multi-source data processing and metadata management are carried out; constructing distributed data association and indexes, including data standardization preprocessing and cross-source data association; the storage performance is optimized and expanded, wherein density sensitive blocking and layered compression are carried out. According to the method, redundant storage of a high-latitude region is greatly reduced, and dynamic adaptation of a local encryption grid is supported. According to the invention, open circulation and collaborative utilization of interdisciplinary data resources are facilitated.

The invention provides an intelligent data query method based on a natural language, and relates to the technical field of intelligent data processing and natural language interaction.The intelligent data query method comprises the steps that firstly, enterprise original data is subjected to standard treatment, and a standardized theme database and a data directory and index definition document are constructed; key semantics are extracted based on unstructured knowledge, and a domain knowledge vector library is fused and constructed by combining document text fragments and vector representation of a mapping relation between historical questions of a user and an SQL (Structured Query Language). And after receiving a natural language question of a user, calling a large language model to identify a task type, and distinguishing knowledge questions and answers, data query and complex analysis. Executing corresponding operations according to different types: directly retrieving a vector library by knowledge questions and answers to generate answers; extracting keywords in data query and generating a query request in combination with context; and in the complex analysis, predefined workflow is judged and executed or intelligent agent processing is called, and query or analysis requirements are output.

The invention provides a distributed storage and data warehouse permission synchronization method and system, and relates to the technical field of permission management, and the method comprises the steps: analyzing user operation data of distributed storage, training a machine learning model, and generating a directory permission intelligent mapping table; analyzing the structure information of the data table in the data warehouse, and generating a multi-dimensional table authority evaluation model; generating a self-adaptive permission rule base by combining the two, and deploying the self-adaptive permission rule base to each node of distributed storage; when the permission is changed, the system calculates an influence range according to the self-adaptive permission rule base and generates a permission updating instruction sequence; and the system monitors the instruction execution state, identifies abnormal nodes, and performs repair based on the permission propagation tree to ensure the permission consistency. According to the method, automatic synchronization of distributed storage and data warehouse authority can be realized, the authority management efficiency is improved, the manual intervention cost is reduced, and the data security is enhanced.

The invention discloses a park data exchange intelligent management system based on block chain and computing network fusion. The system comprises a directory chain subsystem, a data probe subsystem, a computing network fusion scheduling subsystem, an auditing subsystem, an intelligent contract subsystem, a computing network cooperative processing subsystem, a security sand box subsystem and a trusted exchange subsystem. The directory chain subsystem is responsible for receiving data directory information and carrying out uplink evidence storage to ensure that the data directory cannot be tampered; the auditing subsystem completes data access permission auditing according to the computing power distribution scheme and the access strategy; the intelligent contract subsystem generates an intelligent contract containing data processing logic and a computing power scheduling strategy, and triggers computing power resource scheduling; the computing network cooperative processing subsystem coordinates edges and cloud computing resources to complete cooperative computing; the security sandbox system protects data privacy through multi-stage isolation and encryption; and the trusted exchange subsystem safely pushes an encryption processing result and releases computing power resources after the encryption processing result is completed, so that the security and efficiency of data processing are improved.

The invention discloses a context learning-based database query generation method and system and a storage medium, and the method comprises the steps: extracting keywords and phrases from a natural language query, retrieving values similar to the keywords in a database, and returning values most related to the keywords and corresponding columns thereof, retrieving description information related to the keywords and the phrases in a database directory; performing mode selection aiming at the most relevant value of each keyword and the corresponding column, namely screening out the column irrelevant to the natural language query, further selecting a table necessary for generating the SQL query, and finally reducing the mode to a minimum column set necessary for generating the SQL query; and correcting logic and grammar errors of the method model through the database mode, the natural language query, the generated candidate SQL query and the execution result thereof. According to the method, the efficient SQL query statement is generated by intelligently understanding the data context and the query requirement.

The invention discloses a permanent incremental backup method based on an S3 bucket-level virtual snapshot, and relates to the technical field of data storage and backup, the method comprises the following steps: initializing a backup system and starting a backup agent program, an index management service and an S3 object storage service; complete backup is executed, and a VID and S3 bucket-level virtual snapshoot are generated; efficiently managing a mapping relation between a file path and a snapshot identifier through a B + tree and an inverted index by utilizing a MongoDB index service; incremental backup is executed, and file changes are accurately recognized through Hash comparison and modification time comparison; a storage position is optimized by adopting an incremental storage and deduplication storage mechanism and an IncDSI optimization algorithm; executing backup chain integration and cleaning; and performing data recovery based on the VID of the file and the directory. According to the method, the backup efficiency is remarkably improved, the storage overhead is reduced, the data recovery process is optimized, and the method is suitable for data backup and management of a large-scale file system and has important application value and prospect.

The invention discloses a 3DTiles data storage method and a map data query method.The 3DTiles data storage method comprises the following steps that (1) an LOD hierarchy pyramid type hierarchy table is established, and an LOD range index is created; 2) generating a three-dimensional object into an agent point, and segmenting a tile according to a KD-tree structure to construct a three-dimensional KD-tree index tree; and 3) storing the data of the KD-tree leaf node into a GridFS of MongoDB multi-mode storage, mapping a file path into a unique hash value, writing the unique hash value into a corresponding node according to a KD-tree node storage structure, and creating a geographic spatial index. The method creatively solves the problems that file system management is difficult due to the fact that the number of files is large, a hierarchical structure based on a file directory is difficult to support efficient spatial range retrieval, and the spatial query response time is reduced to the millisecond level in combination with a three-dimensional KD-tree index and a MongoDB geographic spatial index.

The invention relates to the technical field of artificial intelligence, can be applied to business scenes of financial science and technology, medical health and the like, and discloses a task scheduling execution method, device, equipment and medium based on intention recognition, and the method comprises the steps: fusing an interaction request and a dialogue history library to generate a session context, and carrying out the analysis to obtain an intention recognition result; task planning is carried out based on the intention recognition result and a knowledge base, and a task subitem set and an execution sequence are generated; matching the task subitem set with a resource directory to generate an execution resource list; scheduling the task subitem set by using the execution sequence to generate an ordered task sequence; generating an execution plan in combination with the ordered task sequence and the execution resource list; and triggering execution based on the execution plan, and aggregating output to generate an execution result and a process tracking record. According to the method, automatic connection of session analysis, task planning, resource matching, sequential scheduling and execution triggering is achieved, manual operation splitting among multiple systems is eliminated, and task processing efficiency and execution accuracy are improved.

The invention provides a many-core cache consistency system, a many-core cache consistency method, electronic equipment, a storage medium and a product, and the system is based on a network-on-chip architecture, and a plurality of nodes, corresponding routers and distributed directories deployed in the routers are introduced. Each node comprises one or more processor cores and a distributed last-stage cache, and a memory address is divided into a plurality of address blocks and is associated with the distributed last-stage cache through a dynamic address mapping rule. The distributed directories and the distributed last-stage caches are in one-to-one correspondence and used for recording the states of cache lines, the router is responsible for receiving and responding to requests from local or remote nodes, querying the distributed directories and updating the states of the cache lines, and efficient management of cache consistency is achieved. The technical scheme belongs to the technical field of communication, effectively solves the problems of high load and communication delay of a main node in a traditional cache consistency protocol, and improves data consistency and access efficiency in a multiprocessor system and a large-scale parallel computing environment.

The present invention relates to the technical field of computers, and provides a log processing method and system, a log management platform, and an electronic device. The method comprises: acquiring real-time log data of each service system, and storing the log data in a message queue; according to a preset time window, writing the log data in the message queue into a target file of a specified storage directory; and in response to completion of data writing of a current time window, on the basis of the storage directory of the target file, calling a log analysis interface of a log analysis service to perform log analysis on the target file. The log processing method in the present invention reduces the risk incident rate of service systems, and improves the localization accuracy of an anomaly condition of a service log.

The invention discloses a metadata query method based on a knowledge graph, which relates to the technical field of data query, and comprises the following steps: collecting metadata from various data sources; then respectively performing knowledge extraction on the data, wherein the knowledge extraction comprises entity extraction, relationship extraction and attribute extraction; constructing a data asset directory according to the metadata after knowledge fusion and a knowledge extraction result of the metadata; then knowledge modeling is carried out, and a knowledge graph is established; and finally, constructing a retrieval enhancement generation process by utilizing a large language model, and retrieving the metadata. According to the method, the natural language can be understood and generated by utilizing the large language model and the retrieval enhancement generation process, and the RAG process retrieves related knowledge and inputs the related knowledge into the language model as a context, so that the processing capability of the model on complex queries is enhanced. According to the method, the extraction accuracy and efficiency can be improved, the method can better adapt to new fields and data types, and the requirement for manual intervention is reduced.

The embodiment of the invention discloses a multi-agent task collaboration method and device and a medium, and the method is characterized in that the method comprises the steps: registering functions corresponding to all agents to a dynamic service directory through an MCP protocol; disassembling the to-be-executed task to obtain a plurality of to-be-executed sub-tasks, matching each to-be-executed sub-task with the capability range of each agent in the dynamic service directory, and establishing a communication channel between each to-be-executed sub-task and the corresponding agent; performing dynamic routing distribution on the to-be-executed sub-tasks through the cooperative bus, and selecting an optimal execution node according to the real-time requirements of the sub-tasks and the node load state of each corresponding agent; according to a subtask dependency relationship defined by a directed acyclic graph, determining an execution mode of each subtask to be executed; and executing each to-be-executed sub-task at the optimal execution node based on the communication channel and the execution mode of each to-be-executed sub-task.

The invention discloses an information scanning and penetration test path planning method in a Windows domain based on reinforcement learning. The method comprises the following steps: carrying out information detection on a target Windows domain environment, surveying and mapping a network topology of the target Windows domain environment, establishing connection with a target machine by utilizing protocols such as LDAP (Lightweight Directory Access Point) and the like, and collecting related information in the domain; the method comprises the following steps of: collecting, sorting and abstracting detected related information, solving a multi-stage sequence decision problem by utilizing a Markov decision process, realizing environment simulation, constructing a reinforcement learning agent, generating a penetration test path by performing interactive training with a simulation environment, and performing adjustment according to a reward strategy so as to obtain a penetration test result. And searching an optimal penetration test path by accumulating rewards, constructing a Windows domain penetration test model, and finally generating the optimal penetration test path. According to the method, collection of information in the Windows domain and planning of the penetration test path are realized, and finally, the environmental safety of the Windows domain is evaluated and maintained.

The invention discloses a focused mode context retrieval enhancement method and system in a knowledge retrieval system, and the method comprises the steps: carrying out the format conversion and preprocessing of a knowledge document, segmenting a text through a focused overlapping partitioning algorithm, and extracting structural features through a natural language processing technology to generate a directory; the text blocks are converted into semantic vectors through a BERT-Context model, and the semantic vectors are stored in a vector library; during retrieval, text blocks are quickly positioned according to question similarity, a long context is formed through splicing, similarity ranking is calculated, cue words are generated by high-correlation text blocks and questions together, and answers are obtained by inputting the cue words into a generation model. According to the method, the problems of semantic deficiency, incomplete context information and undetailed user retrieval context in the existing knowledge retrieval system can be effectively solved, the accuracy and integrity of knowledge retrieval are improved, and particularly, the method is excellent in performance when processing problems in complex fields.

The present application discloses a distributed database-oriented compensating transaction implementation method. The method specifically comprises: a client initiates a transaction request, and determines, by means of an execution directory, whether a transaction SQL exceeds a limit value (S1); a coordination node allocates a timestamp to a transaction initiated by the client in step S1, and simultaneously sends a related operation of the transaction to a data node (S2); upon receiving the transaction request, the data node determines whether to commit or roll back the transaction, and returns an execution result to the coordination node for confirmation and verification, and if the verification fails, the coordination node notifies the data node and performs a compensation operation (S3); and when the operation is completed, the coordination node commits the transaction, performs regular cleaning, and returns the execution result to the client (S4).

The invention belongs to the technical field of data processing, and particularly relates to a method and system for document content recognition and intelligent paragraph and chapter disassembly. The method comprises the following steps: S1, splitting a PDF document into pages, and converting the pages into pictures; s2, performing DOM (Document Object Model) structure analysis on the split PDF page to obtain a text character set; s3, performing layout analysis on the PDF page picture to obtain a layout region set; s4, aggregating the obtained text character set and the layout area set into character blocks; s5, performing reading sequence recovery operation on the character blocks; s6, defining a title matching mode set; s7, traversing the row sequence of the block sequence, and obtaining a hierarchical relationship between directories through a comparison algorithm; s8, outputting a structured content block result; and S9, carrying out visual display on the PDF page picture and the content block coordinates, drawing a new coordinate area by a user, and carrying out secondary disassembly according to the provided coordinate area information.

The invention discloses a method, a product and equipment for sharing a file system by multiple hosts and a storage medium, and relates to the technical field of computers, which comprises the following steps: configuring a shared memory device as a character device, directly mapping to a user mode virtual address space, bypassing a kernel page cache, and directly reading and writing a physical address of the shared memory. A shared memory is divided into a super block area, a metadata area, a log area and a data area during formatting, all hosts perform unified operation through the shared metadata area, stand-alone cache interference is avoided, a hidden metadata file is created to map a non-data area during mounting, a log item is scanned to reconstruct a directory file structure, and the mounting efficiency is improved. All hosts generate a directory file structure completely consistent with the shared memory through log replay, the problem of data consistency caused by a cache strategy in a traditional multi-host file system is solved, and the effects that multiple hosts share a unified memory view, operation is real-time and synchronous, and data access delay is reduced are achieved.

The invention discloses a cross-domain workflow scheduling system for dynamic resource arrangement, and the system comprises a resource abstraction subsystem which is used for collecting all resource platform information and network topology information, and constructing a resource directory and all adaptation layers; the task granularity division subsystem is used for receiving a target task, determining task characteristics of the target task, the cost of each task and an initial dependency graph, determining a target granularity according to the cost of each task, and optimizing the initial dependency graph to obtain a target dependency graph; the workflow arrangement subsystem is used for determining an initial execution plan according to the target dependency graph and the resource directory, determining a corresponding adaptation layer after each target execution platform receives target data, and submitting a corresponding subtask to each target execution platform; and the data preheating management subsystem is used for determining the transmission priority of the cross-platform sub-tasks and sending the target data to each target execution platform in combination with the network topology information, so that the effect of uniformly managing different platform resources is realized, and the stability of the cross-domain tasks is improved.

The invention discloses a PDF (Portable Document Format) document content processing method and device, equipment, a storage medium and a program product, and relates to the technical field of document structured processing. Preprocessing the PDF document to obtain a to-be-processed data set corresponding to each page of the PDF document; determining the page type of each page of the PDF document based on all the to-be-processed data sets and the image of each page of the PDF document; and based on the to-be-processed data set corresponding to each directory page and the image of the directory page, extracting a hierarchical structure relationship of each title data in the directory page, and constructing a directory tree. And matching the title data of the directory page and the title data of the non-directory page based on the semantic similarity and the text similarity between the title data of the directory page and the title data of the non-directory page, and correspondingly filling the content data under each title node of the directory tree according to a matching result to obtain a structured representation result of the PDF document. According to the method and the device, the semantic reduction degree and the structural quality of the PDF document are improved.

The invention discloses a small file distributed high-concurrency aggregation storage method and system, and the method comprises the steps: collecting performance index data, and transmitting the data to a resource coordination server; calculating a queuing rate; determining a target server according to the queuing rate, the file reading concurrency and the disk residual space; determining a first-level directory hash value; selecting a first-level storage directory and a second-level storage directory according to the first-level directory hash value; creating a target aggregation file in the secondary storage directory, generating a 2-byte file format field according to the compression identifier or the non-compression identifier, and generating a 4-byte file length field according to the size of the to-be-written file; writing the file format field, the file length field and the to-be-written file into a target aggregation file in sequence, and recording the initial offset of writing; and generating a file name containing the storage position information, and returning the file name to the client. According to the invention, the efficiency and effect of small file storage are optimized.

The invention belongs to the technical field of office automation, and particularly relates to an automatic file quality inspection method, device and equipment, a medium and a product. The method comprises the following steps: acquiring an archive to be subjected to quality inspection, wherein the archive to be subjected to quality inspection comprises electronic archive metadata and an electronic archive image; determining an archive management type according to the quality inspection subject item list; according to the archive management type, obtaining a verification rule corresponding to the archive management type according to an archive verification directory; according to the archive checking catalog, extracting key contents required by checking from the electronic archive metadata and the electronic archive image; and verifying the key content according to a verification rule. Based on the method, automatic quality inspection of the electronic file metadata and the electronic file images is achieved, and meanwhile the method is easy to expand and high in universality.

The invention discloses a data collaborative directory management method and system, and relates to the technical field of government affair informatization, and the method comprises the steps: generating a directory snapshot containing a global hash value; calculating a directory node hash value of each edge directory node based on the directory snapshot identifier; eliminating clock drift interference through time sequence alignment and dynamic tolerance filtering; inputting the Hash difference time sequence into an isolated forest model to judge a substantial change node; based on the difference entry number and the historical calling weight, combining a dual-threshold rule and an online dichotomy model to hierarchically synchronize requirements; according to a grading result, matching an incremental push mode or a full pull mode, and constructing a synchronous transaction context containing an exponential backoff retry mechanism; synchronous operation is executed through the two-stage state model, and compensation rollback is triggered when the synchronous operation fails; and calculating a health index of the substantially changed node, dynamically selecting a self-healing action and optimizing system parameters. The problem of misjudgment caused by time sequence drift is effectively solved, the synchronization efficiency is improved, and the consistency of directory versions is guaranteed.

The invention relates to the technical field of knowledge base construction, in particular to a knowledge service system construction method, which comprises a knowledge acquisition step for collecting knowledge data from knowledge sources of multiple fields; the preprocessing step is used for preprocessing the knowledge data; the classification step is used for classifying the obtained knowledge data; the similarity calculation step is used for calculating the similarity between different categories of knowledge bases; the first judgment step is used for judging whether the similarity is greater than a first threshold value or not; the association step is used for establishing association between the knowledge according to the similarity; the second judgment step is used for judging whether the similarity is greater than a second threshold value or not; the labeling step is used for constructing a new upper label; a label directory generation step for establishing a label directory; a query demand acquisition step for acquiring a knowledge retrieval demand of a user and generating a query request; the knowledge retrieval step is used for conducting knowledge recommendation according to the query request. The method has the effect of improving the searching efficiency.

The invention discloses a long text analysis method and device, a storage medium and a terminal. Identifying at least one title in the to-be-analyzed long text, and generating directory information of the to-be-analyzed long text based on each title; dividing the long text to be analyzed into a plurality of text paragraphs according to the titles and the input length requirement of the vector model; determining a target superior directory corresponding to each text paragraph from the directory information, and respectively integrating each text paragraph and the target superior directory corresponding to each text paragraph; and inputting the integrated text paragraphs into a vector model to obtain vector features output by the vector model for the text paragraphs. Due to the fact that the whole document is divided into the text paragraphs through the title of the long text, the text paragraphs are divided more reasonably. And continuously integrating the long text information according to the context information contained in the superior directory, so that the semantic information of the text paragraphs is richer, and the model can more accurately understand and process the text paragraphs on this basis.

The invention relates to the technical field of data processing, and provides a question and answer method and system based on paragraph splitting and a knowledge engine platform, and the question and answer method comprises the steps: carrying out directory and title recognition on an uploaded document, and generating a paragraph frame structure with a hierarchical relationship of the document; splitting the text content of the document according to the paragraph frame structure to obtain a document paragraph structure dictionary containing the corresponding text content; according to the document paragraph structure dictionary, text blocks are generated, and a knowledge base is constructed; performing text block vectorization on the text block by adopting a text vectorization model to generate a sparse vector and a dense vector, and storing the sparse vector and the dense vector in a knowledge base; performing keyword extraction and text vectorization processing on the obtained question to obtain a keyword list, a sparse vector and a dense vector of the question; retrieving the knowledge base, and selecting a plurality of text blocks most similar to the problem; and based on the to-be-queried question, a preset cue word and a plurality of text blocks most similar to the question, obtaining a question answer by adopting the large model.

The invention discloses a cross-domain identity authentication and access authorization method, system and device and a medium, and belongs to the field of Internet of Things. According to the method, a unified identity authentication mechanism and a dynamic authority control model are constructed based on a decentralized identifier and a block chain technology. Firstly, an Internet of Things entity submits decentralized identifier information through an identity authentication request; the system generates identity metadata according to the request and records the identity metadata to an identity registry, and meanwhile, constructs a decentration identification document and stores the decentration identification document to a verifiable data registry. And then, the authorizer executes point-to-point trust evaluation based on the identity data and the on-chain traceability information, and generates a verifiable certificate with an encrypted signature. And the verifier completes certificate verification through an intra-domain or cross-domain verification mode, and dynamically grants the authority according to the trust value and the access policy directory. Cross-domain trust migration and dynamic permission adjustment are supported, and the identity authentication security and the access control flexibility in the multi-domain environment of the Internet of Things are improved.