51 results about "Trie" patented technology

Techniques for constraining a language model's generation / decoding process using a finite state machine are described. A finite state machine may represent information corresponding to APIs, arguments and argument values that are available / supported by the system. In some embodiments, a language model (LM) is constrained to generate tokens representing valid API calls based on the finite state machine. The system may enable generation of argument values from a defined set or free-form generation of argument values. The system may also enable unconstrained generation of a response by the LM. In some embodiments, a trie data structure is used to determine the possible next tokens that the LM can generate from.

The invention discloses a large language model reasoning acceleration method and device based on two-stage speculative decoding and a storage medium, and the method comprises the steps: constructing and initializing a Trie tree, and inserting a historical corpus, and phrase sequences in a document library or a code library into the Trie tree one by one; in the reasoning process, longest prefix matching is carried out based on a Trie tree, and a candidate draft sequence is generated by adopting branch backtracking and recursive search; performing confidence evaluation on the candidate draft sequence, calculating a joint confidence score of the sequence through probability multiplication and a Top-K screening mechanism, and judging whether the joint confidence score reaches a confidence threshold; if the accumulated confidence of the candidate sequence reaches a threshold value, skipping a small model generation stage, and directly entering large model verification; otherwise, entering a small model draft completion stage; and the final large model takes the replaced and updated draft sequence as final output. According to the method, adaptive acceleration of the decoding process can be realized, and the long text reasoning delay of the large language model is remarkably reduced while the generation quality is ensured.

Techniques for constraining the results of a generative language model to valid information using knowledge-grounded documentation. A generative language model may generate invalid results, including compound entities and incorrect entity relations. The techniques include, for a given user inquiry, determining a set of documented information, from a particular knowledge base, that corresponds to the user inquiry. The techniques further include determining a subgraph from a knowledge graph representing the knowledge base, as well as determining a trie data structure representation of the set of documented information. The user inquiry and subgraph are provided as input to a trained generative language model for generating a response to the user inquiry. The techniques include using the trie data structure to validate that the generated response corresponds to real information from the set of documented information.

The invention discloses a multi-modal fusion short message compliance and security dual auditing method and system, and the method comprises the steps: encrypting account information through employing an irreversible algorithm based on SHA256 and a random salt value, and decomposing the content of a short message into a text stream, a link stream and a symbol stream through employing a regular expression; traversing each character in the short message content by adopting a prefix mode based on a Trie tree in combination with an AC automaton algorithm to detect sensitive words; performing symbol semantic classification mapping and analysis on the special symbol feature data extracted from the symbol stream; carrying out sending behavior analysis on the text feature data extracted from the text stream; performing special detection on link feature data extracted from the link stream; based on the factors corresponding to the sensitive words, the semantics, the behaviors and the links and the weights of the factors, a multi-modal feature fusion decision risk assessment algorithm is adopted to output a risk score value and a risk decision rule of the risk score value so as to execute short message interception operation or short message release operation. According to the invention, full-dimension perception and dynamic defense of risks can be realized.

The present application relates to a combined word matching detection method and system and a medium. The method comprises: acquiring a preset first combined word, splitting the first combined word to obtain a plurality of first word elements, and, according to the first word elements and the first combined word, constructing a first relationship mapping table; according to the first relationship mapping table, generating a trie; acquiring a text to be subjected to detection, performing, according to the trie, word element matching on said text to obtain a matching record list, and performing aggregation processing on the matching record list to obtain a combined word index dictionary; determining a word element minimum interval of the combined word index dictionary, and, according to the word element minimum interval, calculating a matching score of each combined word in the combined word index dictionary, so as to obtain a matching score list; and, according to the text length and the combined word length, performing normalization processing on the matching score list, so as to obtain a combined word matching result of said text.

The invention discloses an auxiliary-model-free speculation reasoning method based on knowledge enhancement. The method comprises the steps that firstly, a lightweight external knowledge base is constructed based on a segmented caching strategy; secondly, constructing a multi-branch prefix tree Trie supporting dynamic path prediction and context alignment based on an input token sequence by adopting a step-by-step insertion and structural node multiplexing mechanism; generating a tree candidate token path based on the Trie tree, generating a library candidate token path based on an external knowledge base, combining and de-duplicating to generate a mixed candidate token path, constructing a multi-branch tree sharing a prefix, performing parallel reasoning on all nodes in the multi-branch tree through a large model, verifying and updating the Trie tree, and circularly executing the operation to complete speculative reasoning. According to the method, the reasoning performance in a scene without an auxiliary model is comprehensively improved, and a more efficient and more stable generation capability is provided for large model reasoning in tasks in the fields of medical treatment, law and the like.

The invention relates to the field of data access, and particularly provides an MPT-based block chain database system, a data access method, a terminal and a medium, the system comprises a native MPT storage engine used for realizing an MPT node structure in a disk and a memory, and the node structure comprises branch nodes, extension nodes and leaf nodes; the asynchronous I / O module is used for realizing non-blocking disk operation based on Linux iouring; the file system bypass module is used for supporting direct access to the block device; the versioning concurrency control module is used for realizing lock-free reading through an immutable Trie structure and atomic pointer updating; and the dynamic compression module is used for dynamically adjusting a historical data retention strategy according to the disk space utilization rate. The MPT structure storage engine is realized, basic data support is provided for a high-performance block chain system, serialization overhead is eliminated, and storage efficiency is improved.

This application discloses techniques related to tree data structures capable of storing information indicative of database key codes. A computer system can operate a database. The computer system can store a multi-level tree data structure capable of being used to perform key code lookups for the database. In various cases, the multi-level tree data structure can be stored in system memory as a plurality of sub-tree data structures, each sub-tree data structure comprising a set of linked nodes. A given one of the plurality of sub-tree data structures can be stored in system memory as a respective contiguous block of information. The computer system can access the respective contiguous block of a first particular sub-tree data structure encompassing a particular range of levels in the multi-level tree data structure. The access can be performed without accessing one or more other sub-tree data structures encompassing one or more levels within the particular range of levels.

The application relates to an efficient spatial keyword query method based on association rule mining, and belongs to the field of spatial keyword query. The application comprises a data preprocessing stage, an association rule mining stage, an index construction stage and a spatial keyword query stage. The association rule mining stage selects corresponding frequent item sets based on a depth control materialization strategy. The index construction stage constructs a quadtree index for the spatial part of a data set, and creates a trie and an inverted list for the text part of the data set. The query stage adopts a coarse-grained spatial query, and combines the inverted list and the corresponding materialized inverted list of the frequent item sets to perform retrieval. The application combines association rule mining, materialized inverted lists and spatial keyword query, and can greatly improve the query efficiency of spatial keywords.

The application belongs to the field of natural language processing, and particularly relates to a text key phrase extraction method, a storage medium and a device that integrate inter-sentence correlation, comprising: extracting a nominal phrase in a part-of-speech combination mode; filtering the nominal phrase by combining two Trie trees to obtain a candidate phrase set; calculating a global semantic similarity score of the candidate phrase; clustering each sentence to obtain a sentence cluster containing different semantic information; and sorting the candidate phrases in the sentence cluster according to the global semantic similarity score of the candidate phrase to obtain a key phrase set; the application integrates the inter-sentence relationship in the unsupervised key phrase extraction model based on embedding, improves the accuracy of the model, and constructs different Trie trees to calculate mutual information and left and right information entropy as a candidate phrase extraction method in the key phrase extraction model, thereby reducing the probability of incomplete semantic information appearing in the extracted candidate phrase set.

Systems and methods for performing inference for word or wordpiece tokenization are disclosed using a left-to-right longest-match-first greedy process. In some examples, the vocabulary may be organized into a trie structure in which each node includes a precomputed token or token_ID and a fail link, so that the tokenizer can parse the trie in a single pass to generate a list of only those tokens or token_IDs that correspond to the longest matching vocabulary entries in the sample string, without the need for backtracking. In some examples, the vocabulary may be organized into a trie in which each node has a fail link, and any node that would share token(s) or token_ID(s) of a preceding node is instead given a prev_match link that points back to a chain of nodes with those token(s) or token_ID(s).

The invention relates to the technical field of computers, and discloses a high-load micro-service starting method and system, electronic equipment and a storage medium, and the method comprises the steps: executing a hierarchical preloading mechanism, initializing a connection pool, and building a preset minimum idle connection number; checking algorithm optimization is executed, a client feature Trie tree is constructed, root nodes are initialized, child nodes are inserted in sequence according to character string characters, new nodes are created when the child nodes are missing, and the nodes are marked as word ending when character strings are ended; executing request verification mechanism optimization, and returning an error response or an unauthorized response when verification fails; probe parameter dynamic adjustment is executed, and the survival probe detection interval is dynamically adjusted according to the operation indexes; according to the method and the device, the problems of resource waste, low processing efficiency and insufficient stability in a traditional scheme are solved, and reliable technical guarantee is provided for a high-load micro-service scene.

Described are an encoding method and system for ROAs. The encoding method includes the following steps: given a set of authorized IP prefixes an AS which are maintained with an IP address trie. By specifying a sequence of hanging levels on the IP address trie, it is divided into a set of non-overlapping sub-trees, each rooted at a hanging level. A node on a hanging level uniquely defines a sub-tree rooted at it, whose prefix can be encoded as the identifier of this sub-tree. All authorized prefixes covered by a sub-tree can be encoded into a bitmap of 2h bits, where h is the height of this sub-tree. Thus, the set of authorized IP prefixes of an AS is encoded into several tuples (identifier, bitmap).

Embodiments of the present application disclose a word completion method applied to a search scenario to complete an incomplete word input by a user. The method of the embodiments of the present application is based on an improved dictionary tree, hot words are stored in part nodes of the dictionary tree, in the word completion method, a target node matching the string is searched in the dictionary tree Trie, and at least one completed word is output to the user based on the hot words stored in the target node. The word completion efficiency can be improved, and the user is prevented from being recommended words when inputting a too-short string.

This application relates to the field of PLC programming technology, and in particular to a fast symbol retrieval method for ultra-large multi-station PLC projects. The method involves data parsing, symbol metadata standardization, and fragment preprocessing of the PLC project file to obtain standardized SymbolMeta fragment data. Based on the standardized SymbolMeta fragment data, a three-level hierarchical index architecture is constructed, consisting of station-level hash routing, task-level compressed prefix Trie trees, and symbol-level differential encoded inverted lists. According to user query conditions, station-level fast pruning, task-level Trie tree matching, and symbol-level differential decoding aggregation are performed sequentially within the three-level hierarchical index architecture to generate structured retrieval results. Incremental updates are performed on the three-level hierarchical index architecture based on the PLC project's modification instructions. This solves the problems of high latency, weak fuzzy matching capability, high resource consumption, and delayed updates in retrieval of symbols exceeding 100,000, achieving millisecond-level accurate positioning.

This invention belongs to the field of blockchain technology, specifically relating to a semantic identifier blockchain retrieval method and system for large-scale distributed data. This invention designs a unified semantic identifier model, mapping data records from different sources and in different formats to multi-level scalable semantic paths. It constructs a MIR tree forest, integrating the organization of Trie and the verifiability of Merkle, with each scenario as the root. Only the root hash of each scenario or its aggregate root is maintained in the block header, thereby achieving overall constraint on the state of large-scale off-chain semantic indexes while maintaining lightweight on-chain storage overhead. Simultaneously, this invention designs corresponding index traversal and verification object construction mechanisms around various semantic retrieval modes, enabling clients to verify the correctness of target records with only limited path and node hash information. This provides a new technical path for unified semantic management and efficient verifiable retrieval of multi-source heterogeneous data in the HSB environment.

The invention discloses a ten-million-level domain name rule quick loading method based on distributed compiling, which comprises the following steps of: reading and analyzing a rule file containing an accurate domain name and an extensive domain name, and distributing rule analysis result data into a hash table according to a grouping number to obtain a plurality of rule groups; according to the grouping number, compiling threads with the same number are created to form a compiling thread pool; traversing all the rule groups, allocating a compiling thread for each rule group to execute a compiling task, and performing distributed parallel compiling by using a Hyperscan state machine and a Trie tree; receiving an input domain name character string and determining a rule group to which the input domain name character string belongs; if yes, the Hyperscan state machine of the rule group is used for preliminary matching, and if yes, the Trie tree of the rule group is used for accurate matching. According to the method, the problem that a Hyperscan lightweight interface cannot precisely limit a domain name is solved, and rapid loading of a ten-million-level domain name rule is realized.

An IP address searching method is applied to IPv6 and comprises the steps that a searching data structure tree is built based on a multi-bit Trie tree, and each Trie node internally provided with a real node corresponds to one or more IP prefixes in an IP forwarding table; determining a mappable pipeline level range of each node according to an inverse distance of each node in the data structure tree, a pipeline level where a child node is located, a path length from the node to a root node and a position of a pipeline level where a father node is located; wherein the inverse distance is defined as the maximum distance between the node and all subsequent leaf nodes; dynamically mapping each node into an independent storage resource of the corresponding pipeline level according to the mappable pipeline level range and the storage space condition of each pipeline level; and in each pipeline level, parallelly executing IP address searching according to a node mapping result so as to realize pipeline parallel searching. The method can improve the searching efficiency.

The embodiment of the application belongs to the technical field of natural language processing, and relates to a sensitive word detection method and device, computer equipment and a storage medium. The method comprises the following steps: receiving a sensitive word detection request sent by a user terminal; calling a Chinese character Unicode encoding white list, and performing an interference character clearing operation on input text data according to the Chinese character Unicode encoding white list to obtain pure text data; performing a character merging operation on the pure text data to obtain to-be-matched text data; calling a sensitive word Trie tree, and performing a sensitive word similarity matching operation in a similarity matching chain between the characters of the to-be-matched text data and the root node character of the sensitive word Trie tree as a starting point to obtain matching result data; performing a comprehensive scoring operation on the matching result data to obtain a comprehensive scoring result; and outputting the comprehensive scoring result to the user terminal. The sensitive words in the user input text are automatically and efficiently identified and filtered, so that the safety and compliance of the system are protected.

The present invention belongs to the technical field of knowledge embedding in natural language and discloses a method, system, medium, device, and data processing terminal for knowledge embedding. A Trie tree is constructed based on the words in all background knowledge bases. Given an input sentence, the background knowledge words present in the input sentence are searched using the AC tree algorithm; words of different dimensions are converted into word embedding vectors; a multi-dimensional background knowledge embedding vector is generated; and the input vector obtained by concatenating the word embedding vector and the multi-dimensional background knowledge embedding vector is input into an entity recognition model. The knowledge embedding method provided by the present invention converts knowledge of different types, methods, and lengths into the same vector representation and can be applied to the input of any neural network model. The present invention achieved F1 values ​​of 95.03 and 96.78 on the open source named entity recognition datasets MSRA and Resume, respectively. The knowledge embedding improved the model by 1 to 2 points. The F1 value was also improved on the CoNLL2003, MSRA, and OntoNotes 4.0 datasets.

Method includes obtaining historical document images including text that correspond to different document classes; and generating a dictionary using text of the historical document images. The dictionary includes base words occurring with a greatest frequency in each document class. The base words are extracted from the text of the historical document images and arranged in datasets by a document class, where each dataset includes the base words of a same document class that occur with the greatest frequency within that document class. Trie structure is generated using the base words of the datasets that occur with a greatest frequency in each dataset. The trie structure includes internal nodes including root node and leaf nodes in which keys corresponding to the base words occurring with the greatest frequency in each dataset are respectively stored in predefined order. The trie structure is searchable in the predefined order starting with the root node.

This application discloses a method for generating and decompressing compressed log files, and an apparatus for doing so, relating to the field of data processing technology. Based on obtaining a source log file and a trie, and generating an initial compressed log file, the method identifies the first command-line interface log from multiple command-line interface logs included in the source log file. Then, the log timestamp in the first command-line interface log is determined as the base timestamp and written into the header of the initial compressed log file. Next, each command-line interface log is compressed according to the trie and the base timestamp to generate corresponding log tag values. Finally, each log tag value is written into the data area of ​​the initial compressed log file, and the trie is written into the trie area of ​​the initial compressed log file to generate the final compressed log file. This solves the problem of wasted storage space.

The present application belongs to the technical field of data processing, and particularly relates to a digital archive automatic test paper management method and system based on big data analysis, which comprises the following steps: collecting test paper texts and cleaning, and constructing a double-array Trie tree with word frequency weight and semantic label; adopting positive and negative maximum matching bidirectional scanning, and outputting according to the word segmentation number and the ratio of the word frequency weight to the single word number on the segmentation path; using six types of character position conditional random fields containing stroke and pinyin first letter features to identify unregistered words, replacing inverse document frequency with archive category distribution entropy to calculate weighted TF-IDF, and feeding back and adjusting the weight; using genetic algorithm to group test papers based on weighted TF-IDF clustering, introducing distribution entropy disturbance recombination when the iteration tends to be stable, and using keyword hash value and distribution entropy to splice to form a version primary key. The present application can avoid the test paper grouping algorithm falling into local optimum, improve the scientificity and constraint satisfaction ability of the test paper generation, and realize the anti-duplication and traceable management of the test paper archive.

Provided in the present disclosure are a method for performing a data search in a disk, and a disk data index generation method. The method for performing a data search in a disk comprises: receiving a target query key for target data that needs to be searched for; on the basis of the target query key, querying a multi-level data index for the memory location of the target query key level by level, wherein the multi-level data index is generated during data storage and is obtained by means of compression, and comprises multiple levels of query key indexes, and each query key index is generated on the basis of a binary trie and is used for mapping a next level of query key index or the memory location; and acquiring stored key-value pairs on the basis of the memory location, and searching for a target query value corresponding to the target query key, wherein the target query value is used for indicating the target data.

This invention discloses a method and apparatus for creating a state transition dictionary tree for address recognition, belonging to the field of data processing and address recognition technology. The method is implemented as follows: S1, create an address dictionary database, storing all address information in the database; S2, generate a state transition dictionary tree based on the address dictionary database, with the generated dictionary tree corresponding to the address dictionary database; S3, search for address information based on the state transition dictionary tree: obtain the address information in the address dictionary database based on the matching address name and state value. This invention can improve address recognition speed, reduce storage space, and establish a connection between the address dictionary database and the state transition dictionary tree. The matching results help determine the corresponding address information, which is beneficial for subsequent processing and analysis.

An IP address lookup method applied to IPv6, the method comprising: constructing a lookup data structure tree based on a multi-bit Trie tree, wherein each internal Trie node with a real node corresponds to one or more IP prefixes in an IP forwarding table; determining a mappable pipeline stage range of each node according to a reverse distance of each node in the lookup data structure tree, a pipeline stage where a child node is located, a path length from the node to a root node, and a position of a pipeline stage where a parent node is located; wherein the reverse distance is defined as a maximum distance between the node and all successor leaf nodes thereof; dynamically mapping each node to an independent storage resource of a corresponding pipeline stage according to the mappable pipeline stage range and a storage space condition of each pipeline stage; and performing IP address lookup in parallel in each pipeline stage according to the node mapping result to realize pipeline parallel lookup. The method can improve lookup efficiency.

According to an embodiment, a mining method performed by node devices connected to a blockchain network, includes: acquiring, by a specific node device, a transaction including state change data for changing the state of a specific account and a first block including a first trie node related to specific state data of the blockchain network; generating, by the specific node device, a second block including a second trie node based on the transaction and the first trie node; and generating, by the specific node device, a nonce value for a second node corresponding to a state path from a specific leaf node corresponding to the specific account to a specific root node among one or more second nodes included in the second trie node, and determining whether a hash value derived based on the generated nonce value and the second node corresponding to the state path satisfies a predetermined criterion.

This application discloses a method, apparatus, device, storage medium, and product for adjusting a trie, relating to the field of data storage technology. The method for adjusting a trie includes: after the nodes of the trie are updated, obtaining the occurrence count of each node in the current trie; determining whether there is a node in the trie whose occurrence count is greater than a preset threshold; if there is a node whose occurrence count is greater than the preset threshold, constructing that node and at least one of its parent nodes as a shared node of the trie, and based on the shared node, merging the child nodes of that node to obtain a structurally adjusted trie. This application compresses frequently accessed nodes into shared nodes, realizing shared storage of characters rather than independent character storage, thereby improving the space utilization of the trie, reducing the redundancy of character storage in the trie, and improving the storage efficiency of the trie.

The invention relates to the field of electric power communication scheduling data, and provides an electric power communication scheduling data intention identification method based on confusion degree screening, which comprises the following steps of: collecting related consultation data of an electric power communication scheduling system in the whole year, the data covering but not limited to typical electric power communication scheduling data such as a transmission service, a PCM service, a VPN service and the like; an n-gram language model is constructed by using a KenLM algorithm to calculate a sample perplexiity (PPL), a language probability value is efficiently retrieved through a TRIE structure, and medium and high-quality corpora are screened to improve semantic consistency and expression normalization; and through combination of semantic rule filtering and content deduplication, high-quality data with a more standard structure and no information redundancy are extracted. Then, a Transform model is constructed based on a multi-head self-attention mechanism, context semantic features are extracted, and a final feature matrix is obtained through a multi-layer encoder; finally, intention prediction probability distribution is output through a full connection layer and a Softmax classification layer, and a final scheduling intention label is determined according to a threshold value.

An automaton management method and device, in which a computational node constructs a fractal automaton according to first information, the first information comprises at least one first pattern string, the first pattern string comprises at least one character, a first dictionary tree comprises a plurality of nodes, and if at least one first pattern string comprises at least one universal character, the first pattern string comprises at least one character; the plurality of nodes comprise a first root node, at least one second root node and a plurality of non-root nodes, the first root node is empty, one second root node corresponds to one universal character, and the non-root nodes correspond to one non-universal character; the failure pointer of the first root node or the at least one second root node points to itself, the plurality of non-root nodes comprise a first non-root node, the first non-root node is such as a father node, a brother node or a child node of the second root node, and the failure pointer of the first non-root node points to the second root node. The optimized automaton structure is provided through the method, so that the automaton can support fuzzy matching, and operation and maintenance management is facilitated.