41 results about "UTF-8" patented technology

Disclosed is a method for persisting an offline address book in a Unicode compatible format without changing the underlying file formats, record representations, and low level string comparisons. By storing the text information in UCS Transformation Format-8 (UTF-8), Unicode strings can be represented in 8-bit widths and thus are interpreted as just another multi byte character representation. Additionally the offline address book files can still be efficiently searched using the same algorithms for text searching as long as the search key is converted to UTF-8 first.

Systems and methods are described for efficiently processing, searching and/or rewriting variable width encoded data, such as UTF-8 encoded data, will be described. Embodiments of the systems and methods modify and adapt search algorithms, such as the Horspool and Wu-Manber algorithms, to efficiently process and manage searching of variable width encoded text in large blocks of text, such as text that may be carried via a stream of packets thru a network device, such as an intermediary device.

The invention discloses an embedded type terminal and a UTF-8 and GB2312 code conversion method thereof. The UTF-8 and GB2312 code conversion method based on the embedded type movable terminal comprises the following steps: receiving a request which comes from an application program and converts a GB2312 code into a UTF-8 code; reading a GB2312 code character in the embedded type terminal according to the request; converting the read GB 2312 code character into a Unicode code character; directly converting the converted Unicode code character into a UTF-8 code character; and returning a resultof the converted UTF-8 code character to the application program. Under the environment without an operating system window or linux and the condition without other usable APIs, the UTF-8 and GB2312 code conversion method can solve the problem of character treatment of the communication between the embedded type movable terminal and a background server.

The present invention provides a method, an apparatus and a server for processing a data packet. The method comprises the steps of: generating XMPP (Extensible Messaging and Presence Protocol)-based message data to be transmitted, wherein the XMPP-based message data comprises a message header and a message body; performing self-defined packaging for the message header to obtain a packaged message header; converting the message body into a message body conforming to a preset protocol format; packaging the packaged message header and the message body conforming to the preset protocol format to obtain a new message packet; and transmitting the new message packet. According to the method, the apparatus and the server of the present invention, a communication mode of data protocol packets is adopted, an XMPP data packet is packaged by a protocol header, integrity of data is ensured through length field of the protocol header; in addition, communication quality of different types of data packets is defined, so that communication quality modes of different types of data packets can be ensured; and an original packet body is converted into UTF-8 byte stream for processing, thereby facilitating compression with a compression algorithm.

The embodiment of the invention discloses a UTF-8 (8-bit Unicode transformation format) and ANSI (American national standards institute) code identification method used for identifying and distinguishing whether a file is in a UTF-8 coding mode or an ANSI coding mode. The condition that messy codes are displayed in the file in the process of parsing the file by using a wrong coding mode is avoided. The method in the embodiment of the invention comprises the following steps: S1, acquiring a data stream of the file; S2, storing the data stream in an array in a byte form; S3, judging whether preorder bytes exist in the array, if so, deleting the preorder bytes and executing a step S4, otherwise executing the step S4; S4, judging whether a first byte exists in the array, if so, deleting the first byte and executing a step S5, otherwise executing the step S5; S5, judging whether a second byte or a third byte exists in the array, if so, the coding mode of the file is ANSI, otherwise the coding mode of the file is UTF-8. The embodiment of the invention also provides a UTF-8 and ANSI code identification device.

Method and apparatus for at least one of coding or decoding of data. The method comprising retrieving Extensible Markup Language (“XML”)-Unicode Transformation Format 8 (“UTF-8”) data, confirming XML-UTF-8 data in a proper format converting a prolog located within said XML-UTF-8 data, initializing a tag and attribute lookup table, comparing a current character to a plurality of multi-character patterns, determining whether said current character can be converted to a multi-character pattern in said plurality and Unicode, converting said current character to one of ASCII and Unicode when said current character cannot be converted to said multi-character pattern in said plurality, comparing at least one subsequent character to said plurality of multi-character patterns to determine conversion of at least the current character when said current character can be converted more than one way, determining whether there are more characters.

The invention relates to an automatic archive classification method based on an extreme learning machine. The method comprises a learning stage and a running stage, in the first stage, a preprocessingmodule is needed, and the main function of the preprocessing module is to carry out standardized processing on data and remove information incoherent with a task; A preprocessing module firstly unifies text contents into utf-8 coding format; Filtering the illegal characters by adopting a regular expression matching mode; Carrying out word segmentation and part-of-speech tagging by adopting an ICTCLAS Chinese lexical analysis system; And finally, filtering words which often appear in the text but are not significant to text analysis by adopting a Baidu stop word table. According to the method,the archive content in the text can be accurately understood, an efficient and stable archive dictionary with low dimension can be constructed, and meanwhile high classification precision can be guaranteed.

The invention belongs to the field of natural language processing in computational linguistics, and particularly relates to a method and a device for extracting Chinese names of people and places. The method comprises the following steps: S1, transforming text into an encoding format of UTF-8 (unicode transformation format-8); S2, presetting a text threshold L, judging whether text length T is larger than the threshold L, adopting an extension paragraphing method to paragraph the text and turning to step S3 after paragraphing if T is larger than L, and turning to the step S3 if T is smaller than or equal to L; S3, preprocessing the text to remove dirty data; S4, performing part-of-speech tagging on separate Chinese characters in the preprocessed text, and performing word separation and combination on the tagged separate characters; S5, marking phrases in the text, which are matched with target phrases, and calculating matching results. The method and the device can be widely applied to recognition of named entities in fields of search engines, machine translation, data mining and the like.

The invention provides a method and a system for coding two-dimensional code colors. The method includes the steps: converting characters into a byte array corresponding to utf-8 codes of the characters by a two-dimensional code generator; determining color values corresponding to the characters according to a first preset policy; and drawing the color values in a two-dimensional code data area, forming a two-dimensional code picture and then transmitting the two-dimensional code picture to a calculator by the two-dimensional code generator. A decoder reads and analyzes the two-dimensional code picture to be processed, obtains the color value of the two-dimensional code picture to be processed and informs the calculator, wherein the two-dimensional code picture is stored in the calculator. The calculator inquires a color interval table, obtains the corresponding characters and transmits the corresponding characters to the decoder. The calculator is used for determining the color interval table according to a second preset policy according to the color values corresponding to the characters.

The invention provides a coding method and a system of a two-dimension code, the method comprises the steps of a client-side converting characters into a byte digital group of corresponding to character utf-8 code, confirming the corresponding color value of the characters in accordance with a first preset strategy, portraying the color value in the two-dimension code data region, forming a two-dimension code image and sending the image to a server, the server reading and analyzing the two-dimension code image under process, obtaining the color value of the two-dimension code image, inquiring a color section table, and obtaining the corresponding characters, wherein the server confirming the color section table on the basis of the corresponding color value of the characters and a in accordance with second preset strategy.

The invention relates to the technical field of multilateral languages in program software, particularly to the technical problems existing in unusual multilateral language analysis and display, and discloses a multilateral language analysis method for control script programs. The method comprises the following steps of S200, executing the control script programs one by one; S300, capturing program lines of the script programs one by one, and recording to a TXT file; S400, storing the TXT file content to a first-level cache; S500, converting the TXT file content in the first-level cache into script programs in a byte format line by line; S600, converting bytes according to a UTF-8 format; and S700, converting the converted bytes into the script programs in a character format.

The invention relates to a universal forum text extraction method. The method comprises the following steps that a complete html code of a website is extracted, a webpage coded format is tested, and the webpage coded format is uniformly coded into a utf 8 format; a html label type is analyzed, a DOM tree of a webpage is obtained, title information and div label content containing publishing time information are extracted, and the extracted information is classified to generate a list after useless information is filtered; the data length of the list is calculated, and the information is classified with time as a mark and is output in a formatted mode. The extraction method is high in universality, can be applied to most forums, and can accurately extract corresponding data fields of main posts, replies, titles and posting time and output the corresponding data fields in a formatted mode, so that forum information is better utilized.

The present invention is directed to a method, system, and computer program for improved Unicode encoding (UTF-8C). Specifically, the use of a numeric offset system is employed to reduce coding complexity and to mitigate errors in decoding, as compared to standard UTF-8 encoding. Further, a non-zero null string filter may be used to improve the convenience of internalizing C-strings.

This work PC 1 (migration support device) includes: a character code converting unit (11) that converts an EBCDIK+KEIS code into a UTF-8 code; a program converting unit (12) that converts an input program (22) into an output program (32); an exchange information generating unit (13) which causes the output program (32) to read character data to which the UTF-8 code is allocated, and which generates exchange information E that defines, regarding the read character data, the number of areas on a memory specified by the output program (32) so as to be the same as the number of bytes of a byte sequence expressing the EBCDIK+KEIS code allocated to the character data; and an area storing unit (14) that stores a UTF-8 code allocated to the read character data in the areas the number of which is defined by the exchange information E.

The invention provides a two-dimension code processing method and a two-dimensional code client-side. The method comprises the following steps: the two-dimensional code client-side converts characters into byte digit groups corresponding to unicode transformation format-8 (utf-8) coding of the characters, corresponding color values of the characters are determined according to a first preset strategy, and the color values are drawn in a two-dimensional code data area to form two-dimensional code pictures. The two-dimensional code client-side reads and analyzes the two-dimensional code pictures to be processed, obtains the color values of the two-dimensional code pictures to be processed and inquires a color interval list to obtain corresponding characters, wherein the two-dimensional code client-side determines the color interval list according to the color values corresponding to the characters and a second preset strategy.

The present invention provides a data derivation method, apparatus, and electronic device. If the target export file includes both UTF-8 encoded format data and non-UTF-8 encoded format data, the target export file is split into multiple A record that converts all records in the non-UTF-8 encoded format of the record into a target record in a UTF-8 encoded format to obtain an exported file in UTF-8 encoded format. Through the invention, the record in the non-UTF-8 encoding format in the target export file can be converted into the target record in the UTF-8 encoding format, thereby avoiding garbled characters.

The invention provides a method and a computer device for obtaining a front-end html design table based on an execl template, comprising the following steps: step 10, importing an execl form file; 20,open a file storage space and an execl storage space in that storage space; 30, read an execl form file to obtain a file data stream; Storing a file data stream in the file storage space; Step 40, extracting the form data stream in the file data stream and storing the form data stream in the execl storage space; 50, convert that data stream of the form into an html file; 60, through UTF-8, converting an html file into an html data stream; Step 70, extracting the data in the html data stream by extracting the regular expression, and then converting and cleaning the error code and the random code in the data by de-scrambling the regular expression, and finally obtaining the source code of the html form.

A character data set is compressed with a compression algorithm module of a computer system to generate one or more streams of encoded values. The compression module is configured to compress the character data set with an entropy encoder to generate one or more streams of encoded values with UTF-8 or UTF-16. A code points mapper assigns the encoded values to code points in a Unicode format. A UTF encoder encodes the streams of assigned encoded values.