16 results about "Indexer" patented technology

The invention relates to the technical field of post quantum cryptography, and discloses a multi-algorithm collaborative acceleration system for dynamically reconstructing post quantum cryptography. The system comprises a dynamic parameter library, a password graph indexer, a vectorization scheduling engine and a collaborative execution unit. And after dynamic parameter inventory, the quantum cryptography algorithm operates the parameter set in real time, and feedback update is performed according to the collaborative execution unit. A dynamic cryptographic algorithm index graph is constructed by a cryptographic graph indexer, nodes are algorithm instances, edges are in a cooperative relation, and edge weights are fed back and adjusted according to vectorization scheduling engine path weights. The vectorization scheduling engine carries out vectorization coding on a user password task request to generate a task semantic vector, calculates the similarity between the task semantic vector and an index map node embedding vector, and generates an algorithm selection instruction. And the collaborative execution unit receives the instruction, calls a corresponding algorithm instance for operation, and feeds back the parameter change to the dynamic parameter library. According to the system, dynamic optimization and efficient collaboration of the post-quantum cryptography algorithm are realized, and the adaptability to complex scenes is enhanced.

The invention provides a data indexing method and system and a related device, and relates to the field of cloud computing. The method is performed by a data indexing system, the system comprising a first indexer and a first merger, the first merger and the first indexer being located on different compute instances. The method comprises the following steps: a first indexer generates a plurality of first segment files, packs the plurality of first segment files into a first data packet, and sends the first data packet to a first combiner; the first combiner combines the plurality of first segment files in the first data packet to generate a second segment file, and sends the second segment file to the first indexer in the form of the data packet; and the first indexer stores the second segment files and deletes the plurality of first segment files packaged into the first data packet from the disk. According to the embodiment of the invention, the merge operation is independent, and the merge operation is executed by the independent merger, so that the problem that an indexer delays in processing a write request and a read request due to processing of the merge operation can be avoided, and the user experience is improved.

A web crawler comprises a means of detecting webpage content changes and indexing document using Javascript provided by content indexer to detect changes and notify the content indexer of the change.

A data intake and query system can manage the search of data stored at an external location relative to the data intake and query system using one or more indexers. The data intake and query system can receive data stored at the external location. The data intake and query system can process the data and generate an index using the one or more indexers. The data intake and query system can discard the data and store the index and a location identifier of the external location in one or more buckets. In response to a query, the data intake and query system can identify that at least a subset of the data is responsive to the query using the index and can obtain the at least the subset of the data from the external location using the location identifier.

The application discloses a kind of based on HBase's data synchronization method, device and storage medium, it is related to big data field.The method comprises the following steps: receiving user operation index list corresponding to index update;Receive HBase data and synchronize to public region server;Indexer indexes data in the public region server according to index list, and obtains index data;According to the index list after updating, the index data is filtered, and effective data stream is output, and the effective data stream is written into ES.The application is synchronized to data by public region server, and it is conducive to the index of data, can substantially reduce memory, CPU and thread resource occupation, improve the robustness of indexer component under high load scenario, improve product competitiveness.

The application relates to the technical field of post-quantum cryptography, and discloses a dynamic reconstruction post-quantum cryptography multi-algorithm cooperative acceleration system. The system comprises a dynamic parameter library, a cryptographic graph indexer, a vectorization scheduling engine and a cooperative execution unit. The dynamic parameter library stores a set of real-time running parameters of post-quantum cryptography algorithms, and is updated according to feedback from the cooperative execution unit. The cryptographic graph indexer constructs a dynamic cryptographic algorithm index graph according to the parameters, wherein nodes are algorithm instances, edges are cooperative relationships, and edge weights are adjusted according to path weight feedback from the vectorization scheduling engine. The vectorization scheduling engine vectorization-encodes a user cryptographic task request to generate a task semantic vector, calculates the similarity between the task semantic vector and node embedding vectors of the index graph, and generates an algorithm selection instruction. The cooperative execution unit receives the instruction, calls corresponding algorithm instances to operate, and feeds back parameter changes to the dynamic parameter library. The system realizes dynamic optimization and efficient cooperation of post-quantum cryptography algorithms, and enhances the adaptability to complex scenarios.

The invention discloses a privacy information retrieval method based on homomorphic secret sharing, which comprises the following steps that: 1) a client generates a secret key through a secret key generation algorithm, splits the secret key into a plurality of secret key shares by using a homomorphic secret sharing algorithm, and sends the secret key shares to a plurality of non-collusion service nodes; (2) when information retrieval is carried out, the client encrypts a query index through the homomorphic secret sharing algorithm to generate a ciphertext, and meanwhile, the original query index is divided into a plurality of query shares to be distributed to the service nodes; 3) each service node carries out calculation according to the received key share, the query share and a local ciphertext to obtain a response share matched with the received query share, and sends the response share to the client; 4, the client side reconstructs the received response share through a homomorphic secret sharing algorithm, and a retrieval result corresponding to the query indexer.According to the method, retrieval can be completed on the premise that the query index and retrieval content are not leaked, and efficiency and practicability are remarkably improved.

A system and method for structured data representation of a set of media perspectives, including: an autodata generation system executing on a computer processor and configured to: identify a set of caption data for a set of media items; for each of the set of media items, generate a set of structured data representations of the set of media perspectives, by: (i) generating a prompt including caption data of the media item and definition of a structured data representation of a media perspective, and (ii) executing a large language model using the prompt to generate the structured data representation; and store the structured data representations in a document store; and an indexer service including functionality to: execute an encoder model on the structured data representations to generate a set of embeddings; and store the embeddings in a vector store for execution of semantic search using a vector similarity operation.

Systems and methods are described for improving data availability and / or resiliency of indexers of a data intake and query system. A data intake and query system can index and search large amounts of data using one or more indexers. An indexer can store a copy of the data it is processing, the results of processing the data, or a copy of the data that the indexer is assigned to search, in the shared storage system. In the event an indexer fails or is otherwise unable to search data that it has been assigned to search, a cluster master can assign one or more second indexers to search the data. The one or more second indexers can download the data from the shared storage system.

The application discloses a large-scale face retrieval data set cleaning method and system for an open set, and the cleaning method comprises seven steps of face preprocessing, face detection and key point detection, face classification, face feature extraction, face similarity search, face data merging and face data deletion. The method is based on a multi-threshold, multi-path parallel and multi-round iteration method based on similarity search, and can automatically, accurately and efficiently complete the whole-process cleaning work of large-scale face data without manual participation. The search indexer based on the similarity search adopts an incremental construction method, and the face feature vectors of the same identity ID are only one in the search indexer, so that the face data of the same identity ID can be efficiently placed in the same folder.

This invention discloses a few-shot implicit discourse relation recognition method based on retrieval-based context learning, comprising: S1. A pre-trained model containing a cue learning template T is trained using a positive and negative cue contrast learning method to obtain a discourse relation indexer; S2. Based on the discourse relation indexer, the representation vectors of positive and negative MASK lexical units at each level in the training samples are extracted to construct a positive and negative index database; S3. The top K most similar positive and negative training samples are retrieved; S4. The union of the sub-label set of the predicted label in the previous hierarchical structure of the input argument pair and the sub-label set of the truth label of the positive and negative training samples is taken as the candidate label set for the current level; S5. For each level of the label hierarchical structure tree, a cue learning template t is constructed based on the task instruction, positive and negative training samples, candidate label set, and input argument pair, and input to a large language model for inference to obtain the final implicit discourse relation recognition result.

This application discloses a communication method, platform, and system. The method includes: establishing multiple forwarding nodes in an OpenStack cluster and adding all forwarding nodes in the OpenStack cluster to a Kubernetes cluster; the OpenStack cluster includes multiple OpenStack components, with each OpenStack component corresponding to a forwarding node; deploying a load balancer and multiple etcd databases in the Kubernetes cluster; after an OpenStack component in the OpenStack cluster publishes a message using its corresponding forwarding node, the load balancer distributes the message to the multiple etcd databases in the Kubernetes cluster; a forwarding node corresponding to another OpenStack component in the OpenStack cluster queries messages through a Watch mechanism, and upon finding a message, places it in the Indexer of the Kubernetes cluster for local caching for local application reading. The method leverages the Informer message mechanism of the Kubernetes cluster and the synchronization advantages of the distributed etcd databases to improve communication performance; and utilizes the coexistence of the Kubernetes cluster and the OpenStack cluster on the same physical machine to achieve communication between the Kubernetes cluster and the OpenStack cluster.

The invention discloses a dynamic self-adaptive flow table neural index optimization method, device and equipment. Comprising the steps of obtaining a change rule of a flow table, calculating an offset corresponding to the change rule, and updating an affected weight subset of a neural indexer based on the offset; calculating the space-time access popularity of each rule in the flow table, constructing a decision tree based on the space-time access popularity, and executing rule classification or new sub-domain creation operation according to the newly added rule; and when the flow table rule matching fails, obtaining a failure feedback signal, and optimizing the neural indexer based on the failure feedback signal. By updating the affected weight subsets of the neural indexers, full-amount retraining of the model can be avoided, the adjustment time delay is reduced, and model failure caused by rule addition and deletion is prevented. By executing the operation of classifying or establishing the new sub-domain according to the newly added rule, the sub-domain can fit the rule access characteristic, and the number of the new sub-domain and the redundancy calculation are reduced. And in case of failure, the neural indexer is optimized through feedback signals, prediction errors can be corrected, and the prediction accuracy is improved.

A resource processor segments a resource into content chunks based on dynamically identified boundaries and uses the content chunks to determine a first concept, a second concept, and a relationship, such as by performing concept proximity analysis and / or by analyzing resource metadata. A node generator generates and causes execution of computer-executable code to generate a node triple in a graph structure, the node triple including a first node encoding the first concept, a second node encoding the second concept, and a relationship node. A node indexer uses the node triple to generate embeddings, which are indexed relationally to a first node identifier that correlates to the first concept. An interface module queries the embeddings to identify embeddings associated with the first node identifier that correlates to at least a portion of a query. Updates to the resource cause incremental updates in portions of related nodes and related embeddings.

In a computer-implemented method for configuring a distributed computer system comprising a plurality of nodes of a plurality of node classes, configuration files for a plurality of nodes of each of the plurality of node classes are stored in a central repository. The configuration files include information representing a desired system state of the distributed computer system, and the distributed computer system operates to keep an actual system state of the distributed computer system consistent with the desired system state. The plurality of node classes includes forwarder nodes for receiving data from an input source, indexer nodes for indexing the data, and search head nodes for searching the data. Responsive to receiving changes to the configuration files, the changes are propagated to nodes of the plurality of nodes impacted by the changes based on a node class of the nodes impacted by the changes.