56 results about "Delimiter" patented technology

The invention discloses a tracking data compression method and system, which are used for solving the technical problem that the existing tracking data compression technology mostly depends on a universal lossless compression algorithm for data compression, so that the tracking data compression effect is limited. The method comprises the following steps: acquiring a plurality of historical tracking span data, and preprocessing the plurality of historical tracking span data to generate a plurality of historical universal fields and a plurality of historical local fields; constructing a dictionary and a span retrieval tree by adopting a Hash algorithm according to the plurality of historical universal fields and the plurality of historical local fields; and when multiple pieces of to-be-compressed tracking span data are received, compressing the multiple pieces of to-be-compressed tracking span data by adopting the dictionary and the preset separators based on the path node sequence of the span retrieval tree to generate target compressed data.

The present disclosure provides a method for processing data and service system thereof. The present disclosure comprises a steps for receiving a plurality of data from one or more users, wherein the plurality of data includes text in a source language. The present disclosure further comprises steps for storing the plurality of data in a delimited format, wherein the plurality of data is arranged in one or more rows and one or more columns based on the plurality of information associated with one or more users, wherein the data in the one or more rows and the one or more columns are stored in the form of one or more cells comprising one or more characters. The present disclosure further comprises steps for identifying the cell with largest character count from the one or more rows and selecting the row corresponding to the identified cell. The present disclosure further comprises steps for determining if the source language of the selected row is a non-English text; and in response to determining that the selected row is a non-English text, transmitting the selected row to a data translation engine for conversion from source language to target language.

The invention provides a method and a device for identifying repeated and similar texts, computer equipment and a storage medium. The method comprises the following steps: extracting different characters of a to-be-detected text to obtain a divider set composed of a plurality of dividers; extracting a single segmentation symbol from the segmentation symbol set, or extracting a plurality of adjacent characters comprising the segmentation symbol from the text to be detected, and constructing a segmentation mark; segmenting the to-be-detected text by using the segmentation mark to obtain a character string list comprising at least two segmented character strings; extracting features of each character string in the character string list to obtain a feature list corresponding to the character string list; counting the continuous and same occurrence times of the features in the feature list; and when the occurrence frequency meets a preset frequency requirement, determining whether the to-be-detected text contains repeated similar texts or not according to the character string list. According to the invention, automatic detection and recognition of repeated similar texts in a large-model application scene can be realized.

The present application relates to the field of communications, and provides a data processing method and a related device, which are capable of reducing data transmission bandwidth. The method comprises: a first apparatus acquiring first information, wherein the first information indicates to send a first sequence, the first sequence comprises an inter-frame gap, a first subsequence, and a start of frame delimiter, a length of the first sequence is less than a first threshold, a first byte of the first subsequence is used for indicating a start of a data sequence, and the data sequence comprises the first subsequence and the start of frame delimiter; and sending the sequence.

In an aspect, a method of coding a data stream is provided. In an aspect, the method includes mapping, by a data stream encoder, bits into one or more delimiters for the data stream while limiting a maximum run-length of a same polarity of bits in the data stream to be no more than 5. Each of the one or more delimiters include a zero bit and a one bit appended to an 8b / 10b codeword of one or more 8b / 10b codewords in the data stream. In an aspect, the method further includes transmitting, by a transmitter, the one or more delimiters in the data stream.

Disclosed herein are system, method, and computer program product embodiments for a template XSLT (extensible stylesheet language transformation) based NETCONF (network configuration protocol) data collector. An embodiment operates by sending, by a network management device, a NETCONF request to a first network element and a first network element, where the NETCONF request is generated based on a configuration file corresponding to the first network element and the first network element. The embodiment generates a stylesheet corresponding to the NETCONF configuration file. The embodiment receives a first NETCONF reply message and a second NETCONF reply message, where the first NETCONF reply message is generated by the first network element in response to receiving the NETCONF request, and the second NETCONF reply message is generated by the first network element in response to receiving the NETCONF request. The NETCONF reply message and the second NETCONF reply message are transformed into a delimited data format using the stylesheet.

A system and method guide an Artificial Intelligence engine to determine the semantic and grammatical correctness of a user-expanded sentence in real-time. The sentence validation process involves receiving input from the user, the input includes sentence fragment that the user wishes to expand and user-expanded sentence that the user constructs on the fragment provided. The inputs are broken down into tokens. The word-level tokenization algorithm is used, which identifies tokens by splitting the text into spaces, punctuation marks, and other delimiters. Further, a token comparison algorithm is used to assess the relationship between the sentence fragment and the user-expanded sentence to analyze order and placement. Once the token comparison is complete, a prompt is generated using prompt generator to evaluate grammatical and semantic evaluation of the user-expanded sentence. Real-time feedback is provided to the user based on grammatical and semantic evaluation.

A decoding device comprises a decoding processing unit that decodes a plurality of codewords included in an input codeword sequence while switching a codebook or an analysis tree that is to be used for decoding among a plurality of codebooks or analysis trees. The decoding processing unit determines a codebook or an analysis tree that is to be used for decoding a second codeword next to a first codeword included in the input codeword sequence in accordance with the first codeword and the codebook or the analysis tree used for decoding the first codeword. However, in cases where the first codeword is a delimiter codeword that is a specific codeword, the decoding processing unit determines the codebook or the analysis tree that is to be used for decoding the second codeword to be a specific codebook or analysis tree corresponding to the delimiter codeword regardless of the codebook or the analysis tree used for decoding the first codeword.

The invention discloses an analysis method based on an improved longest common subsequence, and belongs to the field of computer science. The method comprises the steps that firstly, a list LCSMap is created to store a plurality of data structures LCSObject, each LCSObject is used for storing a sequence LCSseq and a row index list lineIds, and the sequence LCSseq is the longest common subsequence LCS of a plurality of log entries; receiving log entries and processing the log entries into a token sequence by using a system separator; pre-filtering the token sequence: searching for a prefix tree, searching for an improved inverted list, searching for an improved cycle, establishing the token sequence, and storing the token sequence in a corresponding data structure LCSObject; if the pre-filtering fails, creating a new data structure LCSObject, and taking a log entry as a sequence LCSseq; according to the method, relatively low analysis time consumption can be kept on large-volume log data, and the accuracy of log analysis is not lost.

The present invention reduces template errors due to log parsing by causing a computer to execute: a parsing procedure for parsing, into partial character strings, a portion that can be parsed into a fixed portion and a variable portion which are included in a log, using as a delimiter a blank, a symbol with no alphabet letters or numbers, or a capital alphabet letter; a determination procedure for determining, for each of the partial character strings, whether the partial character string is a variable portion, on the basis of whether the partial character string includes a number in a prescribed mode, or whether the partial character string is present in a prescribed corpus; and a replacement procedure for replacing a partial character string determined to be a variable portion with a character string indicating the variable portion.

The invention relates to a data storage method, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring original data, and carrying out system coding on the original data to obtain an initial character string; under the condition that an unstable character string is recognized from the initial character string, first separators are inserted into the initial position and the end position of the unstable character string respectively, and a second separator is inserted between any two adjacent characters in the unstable character string to form a target character string; on the basis of a preset separator index rule, the first separator and the second separator, separator index information is generated; and writing the separator index information into a preset index position of the target character string, and writing the target character string containing the separator index information into a preset storage position. By adopting the method, the storage space can be saved and the storage efficiency can be improved.

The invention discloses an automobile electronic control basic software code generation method, device and equipment and a storage medium, and relates to the technical field of automobile production and manufacturing, the method comprises the following steps: constructing a template grammar, and creating a template engine according to the template grammar so as to convert a code template file into an m language file; and compiling a code template file corresponding to the target automobile electronic control software based on the template grammar, and calling the template engine to process the code template file so as to realize code generation. The template grammar is used for realizing definition of separators; the separators comprise code separators, expression separators, meta command separators and others; and others are used for representing that text which is not among the code separators, the expression separators and the meta-command separators is output as well. According to the invention, the generation of the automobile electronic control software code can be effectively realized.

This invention discloses an automatic Syslog log parsing method, comprising: collecting Syslog log messages; removing header information and retaining the message body content, parsing it into a string set, and determining a delimiter; using the delimiter to segment the Syslog log message body content into a string array; confirming whether a key exists in the string array; classifying the strings, if a key exists, classifying the logs according to the key and data length; if no key exists, checking the data type and classifying the logs according to the array length and data type; automatically generating parsing templates for different log categories, using the parsing templates to extract the value corresponding to each field, and performing subsequent field mapping or transformation normalization processing. Using this invention, regular expression parsing of Syslog logs can be generated quickly, saving significant manpower.

The application provides a kind of based on SSE's stream data processing display method, system and related equipment, method includes based on the preset parameter construction and initiates HTTP request, establishes the SSE connection with server;In the data stream of the SSE connection, continuously receive data segment and append to data buffer;The data buffer is scanned and identified pre-defined message separator, to extract one or more independent SSE message from the data buffer and store in message queue;From the message queue, the SSE message is obtained, and the SSE message is split into character sequence and stored in character queue;According to the preset timer, periodically take out single or multiple characters from the character queue, render the character to user interface.The application realizes the smooth typewriter display effect by constructing double-layer queue rendering technology, and improves the response performance of SSE stream data in complex business scenarios.

This invention discloses a hierarchical perception-based method and system for analyzing complex English sentences, relating to the interdisciplinary fields of natural language processing and intelligent education. By defining structural delimiters, sentences are divided into syntactic levels according to the number of effective delimiters, first eliminating invalid parallel structural delimiters; then, a syntactic encoding module completes text segmentation, hierarchical tagging, word embedding, and feature fusion; a hierarchical perception module calculates feature weights and generates target syntactic features; finally, a syntactic decoding module outputs the sentence's main body and modifier decomposition results. The hierarchical framework constructed by this invention fully covers the core modifiers of English, is suitable for the scenario of complex sentences in the College Entrance Examination (Gaokao), and can achieve accurate and lightweight decomposition of all types of complex sentences. It effectively solves the problems of irregular decomposition, poor generalization, and poor adaptability of traditional methods, and the analysis process is clear and teachable.

The present invention relates to the field of computer technology, and discloses a method for associating cigarette carton and strip two-dimensional code information with a multi-point verification function. By means of a better scheme, the head and tail positioning codes of the cigarette carton and strip two-dimensional code information are determined. At the same time, at reasonable verification positions and with better data repair, the rejection of strips caused by the failure of associating strip information due to the existence of unreadable records in the carton code queue and the failure of reading at the carton code verification station is greatly reduced, and the yield is increased. Using a phase insertion delimiter does not require reducing the decoding time.

Provided are a medical information processing device, a method for operating a medical information processing device, and a program which can provide support for creating a new medical text.A sentence input by a user is acquired in response to the input. Switching of an attribute of information described in the sentence is determined as a delimiter of the sentence. A structuration process is performed for each unit delimited by the delimiter. The determination is ended according to an end of creation of a medical text including one or more of the sentences. A result of the structuration process is output.

The invention discloses an end-to-end speech separation algorithm based on a speech language model, and the algorithm comprises the steps: discretizing a continuous audio into a 32-order discrete codebook sequence through a residual vector quantization coder-decoder, and introducing a transcription start symbol lt through an SOT strategy; sOSgt, SOSgt; a special separator is lt; sCgt; and a termination symbol lt; eOSgt, EOSgt; splicing a multi-person voice sequence; extracting audio depth features by using a pre-trained WavLM model, and guiding an autoregression decoder to output a separated zero-order codebook sequence in combination with a cross attention mechanism; predicting a high-order codebook sequence step by step through a non-autoregression model, configuring an independent embedding layer to fuse low-order information, and introducing a task embedding mechanism to optimize modeling; based on a special separator lt; sCgt; and slicing the multi-order discrete codebook sequence, and outputting an independent voice source through an Encodec decoder. According to the method, the intelligibility of voice separation and the audio restoration quality can be effectively improved, the decoding speed is high, the subjective hearing experiment result and the downstream task performance are excellent, the scene that the number of speakers is unknown is supported, and the industrialization application prospect is wide.

Methods, systems, and media for lot classification are disclosed. In one example, a classification system for identifying lot listings receives a description for a listing in a publication system, identifies a string in the listing, identifies a quantity word or digit in the string, and converts an identified quantity word into digit form. A normalized string is tokenized to produce tokens, the tokenizing of the normalized string including splitting the normalized string into a series of substrings using a sequence of delimiters. For each substring, an additional split is performed by separating any digit from any other adjacent character, unless that character is another digit, and maintaining an internal character order of each split substring to produce a flattened list of tokenized tokens.

The invention discloses a diffusion enhancement relation-based triple extraction method, which comprises the following steps of: 1) inputting a Chinese text into a BERT model, and converting the Chinese text into a corresponding index to obtain word embedding information; 2) inputting word embedding information into a triple diffusion model, and performing noise addition, feature fusion and de-noising processing to obtain a predicted triple; 3) inputting word embedding information into the character feature extraction network model, learning semantic information through a multi-layer perceptron and a double-affine attention mechanism in combination with relative position coding and a multi-head attention module, relieving a label imbalance problem by adopting a word embedding disturbance mechanism, and outputting a relation triple matrix; and 4) respectively decoding the triads obtained in the two steps, and taking union sets to obtain a final relation triad. According to the method, the boundary diffusion information and the character-level semantic information are fused, so that the problems that Chinese semantics are complex, natural separators are lacked, small-field data semantics extraction is insufficient, extraction of a traditional decoding method is incomplete, labels are unbalanced and the like are effectively solved, the accuracy and integrity of Chinese relation triple extraction and model stability are remarkably improved, and the method is suitable for large-scale popularization and application. The method is suitable for various natural language processing application scenes such as information questions and answers and search engine optimization.

The application discloses a kind of aspect sentiment quadruple prediction methods and systems for dialogue scene, it is related to deep learning technical field.The method includes: obtaining dialogue set;Using the preset dialogue line segmentation algorithm, dialogue set and the semantic label corresponding to dialogue set are split according to different dialogue lines, generate dialogue line set;Constitute completion gap prompt template;The completion gap prompt template and splicing text are connected by separator, obtain input sequence;Using the sequence generation model trained to encode and decode input sequence, generate target sequence;Target sequence is converted into sentiment quadruple, obtain candidate quadruple set;Candidate quadruple set is merged, and the sentiment quadruple set predicted on dialogue level is generated.The application can guarantee the context continuity of dialogue text semantics, effectively alleviate the length limit of model input text, improve the accuracy of generating quadruple.

The invention discloses a log compression method and device based on structured marking and hybrid coding, and solves the technical problem that the compression rate is obviously low due to an existing log compression method. The method comprises the following steps: acquiring an original log message, and preprocessing the original log message by adopting a predefined separator set and two types of regular expressions to generate a dynamic tag sequence and a static tag sequence; classifying the dynamic label sequence, and outputting a plurality of structured labels and a plurality of unstructured labels; based on the plurality of structured markers, generating a refined skeleton and a simplified sub-marker matrix; generating a dictionary file and a binary coded data file according to the static mark sequence, the plurality of unstructured marks, the refined skeleton and the simplified sub-mark matrix; and compressing the dictionary file and the binary coded data file based on a compression algorithm to generate a complete compressed log file.

A signal transmission method, including: transmitting, by a transmitting end, a frame structure signal; where the frame structure signal includes one or more of: a preamble sequence, a delimiter, a data sequence, control information, or a terminator.

The application belongs to the field of artificial intelligence and data security, and discloses a structured sensitive data detection method and system with adaptive splicing scale, which comprises the following steps: firstly, splicing multiple fields of structured data into a text sequence with a preset delimiter, and then performing character set coding to obtain a character index sequence; then, inputting the character index sequence into an embedding layer and a bidirectional long short-term memory network to extract timing features, generating global dependence features through a self-attention mechanism module, extracting multi-scale local features by using a multi-scale sliding window, generating intermediate features through a gating mechanism, obtaining spatial enhancement features through a spatial attention module, and finally splicing the global dependence features and the spatial enhancement features to obtain fusion features; finally, inputting the fusion features into a classifier to obtain the sensitive data category of the structured data. The method has strong adaptability to splicing scale, rich feature representation, high detection accuracy and good robustness, and is suitable for the privacy information classification scene in structured text.

The invention relates to the technical field of data exchange and transmission, in particular to a data transmission method, device and equipment based on JSON self-description and a medium, the method comprises the following steps: generating a message header based on a JSON format, the message header being used for describing a data structure; to-be-transmitted data is organized into a message body in a plain text format, fields and rows are segmented according to separators defined in a message header, and a nested structure is processed according to a nested structure identifier defined in the message header; splicing the message header in the JSON format and the message body in the text format into a complete message; the spliced message is encrypted and / or compressed and then transmitted; and after the message is received, analyzing the message header, analyzing the message body according to the separator and the field sequence described by the message header, and restoring the message body into the structured data. The data is accurately segmented and processed according to the description of the message header, so that the accuracy and efficiency of data transmission are effectively improved, and the error rate in the data transmission process is reduced.

The application provides a URL-based API asset merging method and system, which comprises the following steps: obtaining host access record log data; based on the access record log data, extracting a URL, using a specific separator to split the URI, and forming a word set array; according to the word set data, sequentially taking two words as a relationship, taking the words as nodes, constructing a word relationship graph, and calculating the out-degree and in-degree of the words; according to the out-degree and in-degree obtained in the foregoing step, selecting words with out-degree or in-degree less than a specified threshold value by specifying the threshold value; based on the words obtained in the foregoing step, screening out URIs containing the corresponding words, and calculating the URIs with high similarity in the full-amount URIs by using minhash; based on the URIs selected in the foregoing step, calculating the support degree of the selected corresponding words, and replacing the words with a wildcard string if the support degree is less than a specified threshold value, so as to realize API normalization. The application solves the technical problems of a large API asset list and high dependence on manual operation experience.

The invention relates to the technical field of computers, in particular to a dynamic bidirectional fragmentation method and system based on an order number embedded user identifier, and the method comprises the steps: configuring the total number of fragmentation, and converting a user ID into a reversible pseudo code through a dynamic offset algorithm; firstly, a 32-bit MD5 hash value is generated, all '0' characters in the hash value are replaced by random capital letters, a generated reversible pseudo code is inserted into the hash value according to a fixed bit sequence, a separator is added to the tail of the reversible pseudo code, and finally an order number is generated; performing fragment routing calculation on the user ID to obtain a target fragment index; according to the method, the storage position of the order can be accurately positioned by analyzing the order number, positioning separators from the order number, extracting reversible pseudo codes formed by characters in a fixed bit sequence, restoring the user ID through inverse operation, and executing fragment routing calculation through the user ID, so that the storage position of the order can be accurately positioned. Accurate and rapid query of orders is realized.

Extractive schema linking includes generating a tokenized schema from an SQL schema, a tokenized natural language question, and tokenized candidates. The tokenized candidates are generated by tokenizing candidates from the SQL schema. Each tokenized candidate is formed by a string of tokens having a first token representing an initial delimiter and a last token representing an end delimiter. Vectorial representations of the tokenized schema, the tokenized natural language question, and tokenized candidates are generated, and transformed vectorial representations generated by processing the vectorial representations through a decoder-only model. Concatenated vectors are generated from the transformed vectorial representations of the tokenized candidates, the concatenated vectors generated by concatenating a first transformed vectorial representation corresponding to the first token with a last transformed vectorial representation corresponding to the last token of each tokenized candidate. Quantitative relevancies of the plurality of candidates from the SQL schema are generated based on the concatenated vectors.

The application is suitable for the field of artificial intelligence, and particularly relates to a named entity recognition method and device based on artificial intelligence, equipment and medium. The application matches a dictionary recognition result of a to-be-processed text in an entity dictionary, including N target entities and M entity types corresponding to each target entity i The real-time effectiveness of the dictionary recognition result is ensured through the real-time updated entity dictionary, and the reliability of the named entity recognition result is improved. The to-be-processed text and the dictionary recognition result are spliced into a target character sequence through a preset separator, a pre-trained named entity classification model is used to classify the target character sequence, and a named entity recognition result is obtained. The generation task of determining the named entity and the actual entity type in the to-be-processed text is converted into a classification task of selecting the actual entity type from the candidate entity types, the difficulty of the named entity recognition of the to-be-processed text is reduced, and the accuracy of the named entity recognition result is improved.

The invention belongs to the field of artificial intelligence and data security, and discloses a splicing-scale-adaptive structured sensitive data detection method and system.The method comprises the steps that firstly, multiple fields of structured data are spliced into a text sequence through preset separators, and then character set coding is conducted to obtain a character index sequence; inputting the character index sequence into an embedded layer and a bidirectional long-short-term memory network to extract time sequence features, generating global dependency features through a self-attention mechanism module, extracting multi-scale local features by adopting a multi-scale sliding window, fusing through a gating mechanism to generate intermediate features, and obtaining spatial enhancement features through a spatial attention module; and finally, splicing the global dependency feature and the spatial enhancement feature to obtain a fusion feature. And finally, inputting the fusion features into a classifier to obtain a sensitive data category of the structured data. The method is high in splicing scale adaptability, rich in feature representation, high in detection precision and good in robustness, and is suitable for a privacy information classification scene in a structured text.