UTF-8 (8-bit Unicode transformation format) and ANSI (American national standards institute) code identification method and device

Abstract

The embodiment of the invention discloses a UTF-8 (8-bit Unicode transformation format) and ANSI (American national standards institute) code identification method used for identifying and distinguishing whether a file is in a UTF-8 coding mode or an ANSI coding mode. The condition that messy codes are displayed in the file in the process of parsing the file by using a wrong coding mode is avoided. The method in the embodiment of the invention comprises the following steps: S1, acquiring a data stream of the file; S2, storing the data stream in an array in a byte form; S3, judging whether preorder bytes exist in the array, if so, deleting the preorder bytes and executing a step S4, otherwise executing the step S4; S4, judging whether a first byte exists in the array, if so, deleting the first byte and executing a step S5, otherwise executing the step S5; S5, judging whether a second byte or a third byte exists in the array, if so, the coding mode of the file is ANSI, otherwise the coding mode of the file is UTF-8. The embodiment of the invention also provides a UTF-8 and ANSI code identification device.

Description

technical field [0001] The invention relates to the field of coding, in particular to a UTF-8 and ANSI coding recognition method and device. Background technique [0002] ASCII is a computer coding system based on the Latin alphabet. It is mainly used to display modern English and other Western European languages. It is the most common single-byte encoding system used today. However, since the ASCII code has only 128 characters, it cannot represent all language characters in the world. Different countries and regions have formulated different standards, resulting in their own encoding standards such as GB2312, BIG5, and JIS. These use 2 bytes to represent various Chinese character extension encoding methods of a character, called ANSI (American National Standards Institute, the standard code of the American National Standards Institute). [0003] UTF-8 (8-bit Unicode Transformation Format, Universal Code) is a variable-length character encoding for Unicode, and it is als...

AI Technical Summary

Problems solved by technology

Although the encoding methods of ANSI and UTF-8 are different, the encoding methods are similar or even have overlapping encoding areas. There is a certain probability that a text file in UTF-8 encoding format will be mistakenly parsed and displayed as an ANSI encoding format file.

If the text file or text data stream is parsed using the wrong encoding method, it

Images