41 results about "Delimiter" patented technology

The present disclosure provides a method for processing data and service system thereof. The present disclosure comprises a steps for receiving a plurality of data from one or more users, wherein the plurality of data includes text in a source language. The present disclosure further comprises steps for storing the plurality of data in a delimited format, wherein the plurality of data is arranged in one or more rows and one or more columns based on the plurality of information associated with one or more users, wherein the data in the one or more rows and the one or more columns are stored in the form of one or more cells comprising one or more characters. The present disclosure further comprises steps for identifying the cell with largest character count from the one or more rows and selecting the row corresponding to the identified cell. The present disclosure further comprises steps for determining if the source language of the selected row is a non-English text; and in response to determining that the selected row is a non-English text, transmitting the selected row to a data translation engine for conversion from source language to target language.

The invention provides a method and a device for identifying repeated and similar texts, computer equipment and a storage medium. The method comprises the following steps: extracting different characters of a to-be-detected text to obtain a divider set composed of a plurality of dividers; extracting a single segmentation symbol from the segmentation symbol set, or extracting a plurality of adjacent characters comprising the segmentation symbol from the text to be detected, and constructing a segmentation mark; segmenting the to-be-detected text by using the segmentation mark to obtain a character string list comprising at least two segmented character strings; extracting features of each character string in the character string list to obtain a feature list corresponding to the character string list; counting the continuous and same occurrence times of the features in the feature list; and when the occurrence frequency meets a preset frequency requirement, determining whether the to-be-detected text contains repeated similar texts or not according to the character string list. According to the invention, automatic detection and recognition of repeated similar texts in a large-model application scene can be realized.

The present application relates to the field of communications, and provides a data processing method and a related device, which are capable of reducing data transmission bandwidth. The method comprises: a first apparatus acquiring first information, wherein the first information indicates to send a first sequence, the first sequence comprises an inter-frame gap, a first subsequence, and a start of frame delimiter, a length of the first sequence is less than a first threshold, a first byte of the first subsequence is used for indicating a start of a data sequence, and the data sequence comprises the first subsequence and the start of frame delimiter; and sending the sequence.

In an aspect, a method of coding a data stream is provided. In an aspect, the method includes mapping, by a data stream encoder, bits into one or more delimiters for the data stream while limiting a maximum run-length of a same polarity of bits in the data stream to be no more than 5. Each of the one or more delimiters include a zero bit and a one bit appended to an 8b / 10b codeword of one or more 8b / 10b codewords in the data stream. In an aspect, the method further includes transmitting, by a transmitter, the one or more delimiters in the data stream.

Disclosed herein are system, method, and computer program product embodiments for a template XSLT (extensible stylesheet language transformation) based NETCONF (network configuration protocol) data collector. An embodiment operates by sending, by a network management device, a NETCONF request to a first network element and a first network element, where the NETCONF request is generated based on a configuration file corresponding to the first network element and the first network element. The embodiment generates a stylesheet corresponding to the NETCONF configuration file. The embodiment receives a first NETCONF reply message and a second NETCONF reply message, where the first NETCONF reply message is generated by the first network element in response to receiving the NETCONF request, and the second NETCONF reply message is generated by the first network element in response to receiving the NETCONF request. The NETCONF reply message and the second NETCONF reply message are transformed into a delimited data format using the stylesheet.

A system and method guide an Artificial Intelligence engine to determine the semantic and grammatical correctness of a user-expanded sentence in real-time. The sentence validation process involves receiving input from the user, the input includes sentence fragment that the user wishes to expand and user-expanded sentence that the user constructs on the fragment provided. The inputs are broken down into tokens. The word-level tokenization algorithm is used, which identifies tokens by splitting the text into spaces, punctuation marks, and other delimiters. Further, a token comparison algorithm is used to assess the relationship between the sentence fragment and the user-expanded sentence to analyze order and placement. Once the token comparison is complete, a prompt is generated using prompt generator to evaluate grammatical and semantic evaluation of the user-expanded sentence. Real-time feedback is provided to the user based on grammatical and semantic evaluation.

A decoding device comprises a decoding processing unit that decodes a plurality of codewords included in an input codeword sequence while switching a codebook or an analysis tree that is to be used for decoding among a plurality of codebooks or analysis trees. The decoding processing unit determines a codebook or an analysis tree that is to be used for decoding a second codeword next to a first codeword included in the input codeword sequence in accordance with the first codeword and the codebook or the analysis tree used for decoding the first codeword. However, in cases where the first codeword is a delimiter codeword that is a specific codeword, the decoding processing unit determines the codebook or the analysis tree that is to be used for decoding the second codeword to be a specific codebook or analysis tree corresponding to the delimiter codeword regardless of the codebook or the analysis tree used for decoding the first codeword.

The invention discloses an analysis method based on an improved longest common subsequence, and belongs to the field of computer science. The method comprises the steps that firstly, a list LCSMap is created to store a plurality of data structures LCSObject, each LCSObject is used for storing a sequence LCSseq and a row index list lineIds, and the sequence LCSseq is the longest common subsequence LCS of a plurality of log entries; receiving log entries and processing the log entries into a token sequence by using a system separator; pre-filtering the token sequence: searching for a prefix tree, searching for an improved inverted list, searching for an improved cycle, establishing the token sequence, and storing the token sequence in a corresponding data structure LCSObject; if the pre-filtering fails, creating a new data structure LCSObject, and taking a log entry as a sequence LCSseq; according to the method, relatively low analysis time consumption can be kept on large-volume log data, and the accuracy of log analysis is not lost.

This invention discloses an automatic Syslog log parsing method, comprising: collecting Syslog log messages; removing header information and retaining the message body content, parsing it into a string set, and determining a delimiter; using the delimiter to segment the Syslog log message body content into a string array; confirming whether a key exists in the string array; classifying the strings, if a key exists, classifying the logs according to the key and data length; if no key exists, checking the data type and classifying the logs according to the array length and data type; automatically generating parsing templates for different log categories, using the parsing templates to extract the value corresponding to each field, and performing subsequent field mapping or transformation normalization processing. Using this invention, regular expression parsing of Syslog logs can be generated quickly, saving significant manpower.

The application provides a kind of based on SSE's stream data processing display method, system and related equipment, method includes based on the preset parameter construction and initiates HTTP request, establishes the SSE connection with server;In the data stream of the SSE connection, continuously receive data segment and append to data buffer;The data buffer is scanned and identified pre-defined message separator, to extract one or more independent SSE message from the data buffer and store in message queue;From the message queue, the SSE message is obtained, and the SSE message is split into character sequence and stored in character queue;According to the preset timer, periodically take out single or multiple characters from the character queue, render the character to user interface.The application realizes the smooth typewriter display effect by constructing double-layer queue rendering technology, and improves the response performance of SSE stream data in complex business scenarios.

This invention discloses a hierarchical perception-based method and system for analyzing complex English sentences, relating to the interdisciplinary fields of natural language processing and intelligent education. By defining structural delimiters, sentences are divided into syntactic levels according to the number of effective delimiters, first eliminating invalid parallel structural delimiters; then, a syntactic encoding module completes text segmentation, hierarchical tagging, word embedding, and feature fusion; a hierarchical perception module calculates feature weights and generates target syntactic features; finally, a syntactic decoding module outputs the sentence's main body and modifier decomposition results. The hierarchical framework constructed by this invention fully covers the core modifiers of English, is suitable for the scenario of complex sentences in the College Entrance Examination (Gaokao), and can achieve accurate and lightweight decomposition of all types of complex sentences. It effectively solves the problems of irregular decomposition, poor generalization, and poor adaptability of traditional methods, and the analysis process is clear and teachable.

Provided are a medical information processing device, a method for operating a medical information processing device, and a program which can provide support for creating a new medical text.A sentence input by a user is acquired in response to the input. Switching of an attribute of information described in the sentence is determined as a delimiter of the sentence. A structuration process is performed for each unit delimited by the delimiter. The determination is ended according to an end of creation of a medical text including one or more of the sentences. A result of the structuration process is output.

The invention discloses an end-to-end speech separation algorithm based on a speech language model, and the algorithm comprises the steps: discretizing a continuous audio into a 32-order discrete codebook sequence through a residual vector quantization coder-decoder, and introducing a transcription start symbol lt through an SOT strategy; sOSgt, SOSgt; a special separator is lt; sCgt; and a termination symbol lt; eOSgt, EOSgt; splicing a multi-person voice sequence; extracting audio depth features by using a pre-trained WavLM model, and guiding an autoregression decoder to output a separated zero-order codebook sequence in combination with a cross attention mechanism; predicting a high-order codebook sequence step by step through a non-autoregression model, configuring an independent embedding layer to fuse low-order information, and introducing a task embedding mechanism to optimize modeling; based on a special separator lt; sCgt; and slicing the multi-order discrete codebook sequence, and outputting an independent voice source through an Encodec decoder. According to the method, the intelligibility of voice separation and the audio restoration quality can be effectively improved, the decoding speed is high, the subjective hearing experiment result and the downstream task performance are excellent, the scene that the number of speakers is unknown is supported, and the industrialization application prospect is wide.

The invention discloses a diffusion enhancement relation-based triple extraction method, which comprises the following steps of: 1) inputting a Chinese text into a BERT model, and converting the Chinese text into a corresponding index to obtain word embedding information; 2) inputting word embedding information into a triple diffusion model, and performing noise addition, feature fusion and de-noising processing to obtain a predicted triple; 3) inputting word embedding information into the character feature extraction network model, learning semantic information through a multi-layer perceptron and a double-affine attention mechanism in combination with relative position coding and a multi-head attention module, relieving a label imbalance problem by adopting a word embedding disturbance mechanism, and outputting a relation triple matrix; and 4) respectively decoding the triads obtained in the two steps, and taking union sets to obtain a final relation triad. According to the method, the boundary diffusion information and the character-level semantic information are fused, so that the problems that Chinese semantics are complex, natural separators are lacked, small-field data semantics extraction is insufficient, extraction of a traditional decoding method is incomplete, labels are unbalanced and the like are effectively solved, the accuracy and integrity of Chinese relation triple extraction and model stability are remarkably improved, and the method is suitable for large-scale popularization and application. The method is suitable for various natural language processing application scenes such as information questions and answers and search engine optimization.

The application discloses a kind of aspect sentiment quadruple prediction methods and systems for dialogue scene, it is related to deep learning technical field.The method includes: obtaining dialogue set;Using the preset dialogue line segmentation algorithm, dialogue set and the semantic label corresponding to dialogue set are split according to different dialogue lines, generate dialogue line set;Constitute completion gap prompt template;The completion gap prompt template and splicing text are connected by separator, obtain input sequence;Using the sequence generation model trained to encode and decode input sequence, generate target sequence;Target sequence is converted into sentiment quadruple, obtain candidate quadruple set;Candidate quadruple set is merged, and the sentiment quadruple set predicted on dialogue level is generated.The application can guarantee the context continuity of dialogue text semantics, effectively alleviate the length limit of model input text, improve the accuracy of generating quadruple.

The invention discloses a log compression method and device based on structured marking and hybrid coding, and solves the technical problem that the compression rate is obviously low due to an existing log compression method. The method comprises the following steps: acquiring an original log message, and preprocessing the original log message by adopting a predefined separator set and two types of regular expressions to generate a dynamic tag sequence and a static tag sequence; classifying the dynamic label sequence, and outputting a plurality of structured labels and a plurality of unstructured labels; based on the plurality of structured markers, generating a refined skeleton and a simplified sub-marker matrix; generating a dictionary file and a binary coded data file according to the static mark sequence, the plurality of unstructured marks, the refined skeleton and the simplified sub-mark matrix; and compressing the dictionary file and the binary coded data file based on a compression algorithm to generate a complete compressed log file.

A signal transmission method, including: transmitting, by a transmitting end, a frame structure signal; where the frame structure signal includes one or more of: a preamble sequence, a delimiter, a data sequence, control information, or a terminator.

The application belongs to the field of artificial intelligence and data security, and discloses a structured sensitive data detection method and system with adaptive splicing scale, which comprises the following steps: firstly, splicing multiple fields of structured data into a text sequence with a preset delimiter, and then performing character set coding to obtain a character index sequence; then, inputting the character index sequence into an embedding layer and a bidirectional long short-term memory network to extract timing features, generating global dependence features through a self-attention mechanism module, extracting multi-scale local features by using a multi-scale sliding window, generating intermediate features through a gating mechanism, obtaining spatial enhancement features through a spatial attention module, and finally splicing the global dependence features and the spatial enhancement features to obtain fusion features; finally, inputting the fusion features into a classifier to obtain the sensitive data category of the structured data. The method has strong adaptability to splicing scale, rich feature representation, high detection accuracy and good robustness, and is suitable for the privacy information classification scene in structured text.

The application provides a URL-based API asset merging method and system, which comprises the following steps: obtaining host access record log data; based on the access record log data, extracting a URL, using a specific separator to split the URI, and forming a word set array; according to the word set data, sequentially taking two words as a relationship, taking the words as nodes, constructing a word relationship graph, and calculating the out-degree and in-degree of the words; according to the out-degree and in-degree obtained in the foregoing step, selecting words with out-degree or in-degree less than a specified threshold value by specifying the threshold value; based on the words obtained in the foregoing step, screening out URIs containing the corresponding words, and calculating the URIs with high similarity in the full-amount URIs by using minhash; based on the URIs selected in the foregoing step, calculating the support degree of the selected corresponding words, and replacing the words with a wildcard string if the support degree is less than a specified threshold value, so as to realize API normalization. The application solves the technical problems of a large API asset list and high dependence on manual operation experience.

The invention relates to the technical field of computers, in particular to a dynamic bidirectional fragmentation method and system based on an order number embedded user identifier, and the method comprises the steps: configuring the total number of fragmentation, and converting a user ID into a reversible pseudo code through a dynamic offset algorithm; firstly, a 32-bit MD5 hash value is generated, all '0' characters in the hash value are replaced by random capital letters, a generated reversible pseudo code is inserted into the hash value according to a fixed bit sequence, a separator is added to the tail of the reversible pseudo code, and finally an order number is generated; performing fragment routing calculation on the user ID to obtain a target fragment index; according to the method, the storage position of the order can be accurately positioned by analyzing the order number, positioning separators from the order number, extracting reversible pseudo codes formed by characters in a fixed bit sequence, restoring the user ID through inverse operation, and executing fragment routing calculation through the user ID, so that the storage position of the order can be accurately positioned. Accurate and rapid query of orders is realized.

Extractive schema linking includes generating a tokenized schema from an SQL schema, a tokenized natural language question, and tokenized candidates. The tokenized candidates are generated by tokenizing candidates from the SQL schema. Each tokenized candidate is formed by a string of tokens having a first token representing an initial delimiter and a last token representing an end delimiter. Vectorial representations of the tokenized schema, the tokenized natural language question, and tokenized candidates are generated, and transformed vectorial representations generated by processing the vectorial representations through a decoder-only model. Concatenated vectors are generated from the transformed vectorial representations of the tokenized candidates, the concatenated vectors generated by concatenating a first transformed vectorial representation corresponding to the first token with a last transformed vectorial representation corresponding to the last token of each tokenized candidate. Quantitative relevancies of the plurality of candidates from the SQL schema are generated based on the concatenated vectors.

The application is suitable for the field of artificial intelligence, and particularly relates to a named entity recognition method and device based on artificial intelligence, equipment and medium. The application matches a dictionary recognition result of a to-be-processed text in an entity dictionary, including N target entities and M entity types corresponding to each target entity i The real-time effectiveness of the dictionary recognition result is ensured through the real-time updated entity dictionary, and the reliability of the named entity recognition result is improved. The to-be-processed text and the dictionary recognition result are spliced into a target character sequence through a preset separator, a pre-trained named entity classification model is used to classify the target character sequence, and a named entity recognition result is obtained. The generation task of determining the named entity and the actual entity type in the to-be-processed text is converted into a classification task of selecting the actual entity type from the candidate entity types, the difficulty of the named entity recognition of the to-be-processed text is reduced, and the accuracy of the named entity recognition result is improved.

Methods in a system for programming in natural language, comprising the steps - Machine capture and analysis (S100) of digital input information, which includes a plurality of input elements in natural language, - Determine (S200), by the system, based on analyzing the input information, an elementary data type of an input element of the input information, and - Generating (S300) source code for the elementary data type, the step of which includes capturing the input information: - Dividing the input information into sentences, where a sentence comprises a sequence of consecutive input elements in the input information and ends with a separator; and where the step of determining (S200) the elementary data type comprises the following substeps: - Recognizing a first set comprising a first sequence of input elements and a second set comprising a second sequence of input elements, wherein the first sequence and the second sequence differ only at one position, wherein the first sequence includes a first difference input element at this position and the second sequence includes a second difference input element at this position, - Determining the elementary data type for the difference input elements; assigning the specified elementary data type to each of the difference input elements.

To provide a decoder and a method for deriving a necessary parameter from an access unit.SOLUTION: A video decoder 20 comprises an in-loop filter 90 which filters a reconfiguration version of a decoded picture, and a parameterizing unit. The parameterizing unit reads in-loop filter control information for parameterizing the in-loop filter from a parameter set located in an access unit of a decoded picture following a video encoding unit along the sequence of a data stream 14 and / or a part of a video encoding unit following data included in a video encoding unit carrying block-based prediction parameter data and predicted residue data along the sequence of the data stream, and also parameterizes the in-loop filter so as to filter the reconfiguration version of the encoded picture by a method corresponding to the in-loop filter control information.SELECTED DRAWING: Figure 2

This application relates to a decoding method, apparatus, and computer-readable storage medium, comprising: determining first delimiter information based on target encoded data; then determining decoding rules corresponding to first encoded text information based on the first delimiter information; wherein, in the text identified by the target encoded data, the delimiter text identified by the first delimiter information and the encoded text identified by the first encoded text information are arranged adjacently, and the delimiter text identified by the first delimiter information precedes the encoded text identified by the first encoded text information; and then, using the decoding rules corresponding to the first encoded text information, the first encoded text information is decoded to obtain first decoded text information. This application can support the decoding of target encoded data generated by mixed encoding of multiple encoding formats, and does not require pre-scanning of the target encoded data, enabling simultaneous scanning and decoding, which greatly improves decoding efficiency.

The invention relates to the technical field of natural language processing, and discloses a T5 and knowledge graph data enhancement-based small sample event extraction method, which comprises the following steps of: segmenting an original text according to a preset segmentation bound character, and screening a longest fragment as a covering object; performing prediction generation and backfilling on the covering object by utilizing the T5 model after fine adjustment to obtain a preliminary enhanced text; performing similar entity replacement on nouns or pronouns except the key information in the preliminarily enhanced text based on a Chinese concept knowledge graph to obtain a new sample; and combining the new sample with the original text to form an enhanced data set, and driving the event extraction model to train. According to the method, text distribution consistency is kept by adopting generative fine tuning, semantic diversity is increased by combining knowledge graph replacement, and meanwhile, tagging information is directly inherited by protecting trigger words and arguments, so that the accuracy and generalization of event extraction under a small sample condition are effectively improved.

Embodiments of the present application disclose a string decoding method and device, electronic equipment and a storage medium. The method comprises: in the case of receiving a to-be-decoded string parameter, traversing each byte in the to-be-decoded string parameter; when any byte is traversed, reading the code value of the byte, judging whether the byte is a high-position Chinese byte based on the code value of the byte, if it is determined that the byte is a high-position Chinese byte, offsetting the traversal pointer by a first preset offset value; if the byte is not a high-position Chinese byte, judging whether the byte is a byte storing a delimiter, if yes, determining the position of the byte in the to-be-decoded string parameter, and offsetting the traversal pointer by a second preset offset value; in the case of traversing a byte storing a preset end character, the to-be-decoded string parameter traversal ends, and the to-be-decoded string parameter is cut into to-be-decoded character units according to the determined position of the preset delimiter; and decoding each to-be-decoded character unit based on a preset decoding rule.

The application discloses a double-screen adaptive translation generation method and system based on semantic processing, relates to the field of double-screen adaptive translation generation based on semantic processing, and is used for improving the display clarity and user understanding efficiency of a double-screen device in a professional field conversation to process an ultra-long translation, and comprises the following steps: acquiring a translation generated by a translation program; identifying abnormal words in the translation; the abnormal words are words with a length greater than a preset threshold and lacking natural delimiters; adjusting visual presentation parameters of the abnormal words, wherein the visual presentation parameters comprise any one or more of the following: character spacing, halo parameters, and font style; and generating the translation according to the adjusted visual presentation parameters of the abnormal words.

A method and system for chunking a document are provided. The method according to some embodiments may include chunking a document including a plurality of sentences into a plurality of section chunks based on sentences including a section delimiter, determining whether a size of each of the plurality of section chunks exceeds a preset first threshold and chunking a section chunk, a size of the section chunk among the plurality of section chunks exceeds the preset first threshold, into a plurality of sub-chunks based on whether a similarity between sentences included in the section chunk is equal to or greater than a second threshold. The second threshold may be determined based on a similarity distribution of a query for the document, calculated by comparing a query generated from the document using a generative model with the document.

System (200) for comparing two strings (11, 11') comprising the following: • a communication unit (210) designed to receive a first and a second input string (11, 11') each specifying a sequence of characters of an alphabet, in particular via an application interface over a network (300), e.g. via the Internet; • a processing unit (220) configured to insert a separator sequence (13) between the characters of the first and second input string (11, 11') to generate a corresponding first and second modified input string (14, 14'), wherein the separator sequence (13) specifies a stochastically rare string; • a tokenization unit (230) configured to tokenize the first and second modified input strings (14, 14') into a first and second token sequence (15, 15') such that a token (19, 19') of the first and second token sequence (15, 15') corresponds to a character of the first and second input strings (11, 11') or of the separator sequence (13), respectively; • a comparison unit (250) trained to compare the first and second input strings (11, 11') using a transformer-based inference model (17) with the first and second token sequence (15, 15'), excluding the tokens (19') corresponding to the separator sequence (13) from the comparison.