File encoding format detection method and device

Abstract

The present application discloses a file encoding format detection method and device. The method includes: obtaining a target encoding file, detecting whether the bytes of the target encoding file include encoding format information; when the bytes of the target encoding file include encoding format information, determining the target encoding format of the target encoding file based on the encoding format information; when the bytes of the target encoding file do not include encoding format information, splitting the target encoding file into a first preset number of target sub-files, using an encoding format detector to detect the first encoding format of each target sub-file, obtaining an encoding format set, and determining the target encoding format of the target encoding file from the encoding format set. The present application solves the technical problem that related detection technologies cannot quickly and accurately identify the encoding formats of different files.

Description

Technical Field

[0001] The present application relates to the field of file processing technology, and in particular to a file encoding format detection method and device. Background Art

[0002] As computer programs process more and more data, some of the data can be directly stored in the server in the form of text files. Since different types of text files can be encoded in different formats, when reading or sending various text files, improper operations are often performed due to unclear encoding formats, resulting in garbled characters in the output files.

[0003] There are two different solutions for detecting file encoding formats in related technologies. One is to determine the encoding format of files and byte streams based on Unicode encoding rules, but this solution can only detect files with standard encodings; the other is to use multiple encoding formats to detect the string of file identification tags, and use search engines to search for each transcoded detection result, but this technical solution relies heavily on search engines, and the code does not have the ability to work independently. Both of the above solutions cannot quickly and accurately identify various file encoding formats.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a method and device for detecting a file encoding format, so as to at least solve the technical problem that related detection technologies cannot quickly and accurately identify the encoding formats of different files.

[0006] According to one aspect of an embodiment of the present application, a file encoding format detection method is provided, including: obtaining a target encoding file, detecting whether the bytes of the target encoding file include encoding format information; when the bytes of the target encoding file include encoding format information, determining a target encoding format of the target encoding file based on the encoding format information; when the bytes of the target encoding file do not include encoding format information, splitting the target encoding file into a first preset number of target sub-files, using a encoding format detector to detect a first encoding format of each target sub-file, obtaining an encoding format set, and determining the target encoding format of the target encoding file from the encoding format set.

[0007] Optionally, the target encoding format includes at least one of the following: Unicode encoding format, Unicode bigendian encoding format, UTF-8 encoding format, UTF-16 encoding format, GBK encoding format, GB2312 encoding format, ISO-8859-1 encoding format.

[0008] Optionally, detect whether the bytes of the target encoding file meet the following conditions: the first two bytes of the target encoding file are FF-FE, or the first two bytes of the target encoding file are FE-FF, or the first three bytes of the target encoding file are EF-BB-BF; when the bytes of the target encoding file meet any of the above conditions, determine that the bytes of the target encoding file include encoding format information, and the encoding format information is the first two bytes or the first three bytes of the target encoding file; when the bytes of the target encoding file do not meet all of the above conditions, determine that the bytes of the target encoding file do not include encoding format information.

[0009] Optionally, determining the target encoding format of the target encoding file based on the encoding format information includes: when the first two bytes of the target encoding file are FF-FE, determining that the target encoding format of the target encoding file is the Unicode encoding format; when the first two bytes of the target encoding file are FE-FF, determining that the target encoding format of the target encoding file is the Unicode bigendian encoding format; when the first three bytes of the target encoding file are EF-BB-BF, determining that the target encoding format of the target encoding file is the UTF-8 encoding format.

[0010] Optionally, the types of all first encoding formats in the encoding format set are counted, and a first quantity of each type of first encoding format is determined; the first encoding formats of each type are sorted in descending order based on the first quantity; and the target encoding format of the target encoding file is determined based on the first two types of first encoding formats.

[0011] Optionally, determining the target encoding format of the target encoding file based on the first two types of first encoding formats includes: when the first encoding format ranked first is not garbled, determining the target encoding format of the target encoding file to be the first encoding format ranked first; when the first encoding format ranked first is garbled, comparing the first number of the first encoding format ranked first with a preset threshold, and when the first number is greater than the preset threshold, determining that the target encoding format of the target encoding file is garbled; when the first number is not greater than the preset threshold, determining the target encoding format of the target encoding file to be the second first encoding format ranked second.

[0012] Optionally, the encoding format detector is an open source cpdetector detector.

[0013] According to another aspect of an embodiment of the present application, a file encoding format detection device is also provided, including: a detection module, used to obtain a target encoding file and detect whether the bytes of the target encoding file include encoding format information; a first determination module, used to determine the target encoding format of the target encoding file based on the encoding format information when the bytes of the target encoding file include encoding format information; a second determination module, used to split the target encoding file into a first preset number of target sub-files when the bytes of the target encoding file do not include encoding format information, use an encoding format detector to detect a first encoding format of each target sub-file, obtain an encoding format set, and determine the target encoding format of the target encoding file from the encoding format set.

[0014] According to another aspect of an embodiment of the present application, a non-volatile storage medium is further provided, the non-volatile storage medium including a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to perform the above-mentioned file encoding format detection.

[0015] According to another aspect of an embodiment of the present application, an electronic device is further provided, comprising: a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to perform the above-mentioned file encoding format detection through the computer program.

[0016] In the embodiment of the present application, the target encoding file is first obtained, and whether the bytes of the target encoding file include encoding format information is detected. When the bytes of the target encoding file include encoding format information, the target encoding format of the target encoding file is determined based on the encoding format information; when the bytes of the target encoding file do not include encoding format information, the target encoding file is split into a first preset number of target sub-files, and the first encoding format of each target sub-file is detected by using an encoding format detector to obtain an encoding format set, and the target encoding format of the target encoding file is determined from the encoding format set. In this way, it is possible to classify and detect various types of files according to whether the bytes of the target encoding file include encoding format information, thereby solving the technical problem that the relevant detection technology cannot quickly and accurately identify the encoding formats of different files. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 It is a flowchart of a file encoding format detection method according to an embodiment of the present application;

[0019] Figure 2It is a structural schematic diagram of a file encoding format detection device according to an embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0021] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0022] In order to better understand the embodiments of the present application, some nouns or terms that appear in the description of the embodiments of the present application are translated and explained as follows:

[0023] Unicode (Universal Multiple-Octect Coded Character Set) encoding: Unicode is a character encoding method developed by an international organization that can accommodate all text and symbols. It uses numbers 0-0x10FFFF to map these characters and can accommodate up to 1114112 characters. Unicode encoding refers to the UCS-2 encoding method, which directly stores the Unicode code of the character in two bytes.

[0024] Unicode big endian encoding: The lowest address is stored in the high byte, which can be called high-order priority, and the memory is stored sequentially starting from the lowest address.

[0025] UTF-8 (8-bit Unicode Transformation Format, 8-bit) encoding: UTF-8 is a variable-length character encoding for Unicode, which can be used to identify any character in the Unicode standard, and the first two bytes in its programming are still compatible with ASCII, so that the original software that processes ASCII characters can continue to be used without or with only a few modifications. Among them, UTF-8 uses 1-4 bytes to encode each character.

[0026] UTF-16 (16-bit Unicode Transformation Format, 16-bit) encoding: UTF-16 is the third layer of the five-layer model of Unicode character encoding, that is, an implementation of the Character Encoding Form. That is, the abstract code points of the Unicode character set are mapped to a 16-bit integer sequence for data storage or transmission.

[0027] GB2312 encoding: GB2312 encoding is the code for the Chinese character set and encoding. It divides the included characters into 94 areas, numbered from area 01 to area 94. Each area contains 94 characters, numbered from position 01 to position 94. Each character of GB2312 is determined by its unique corresponding area code and position number. Among them, ANSI is the default encoding method, which is ASCII encoding for English characters and GB2312 encoding for simplified Chinese characters.

[0028] GBK (Chinese Internal Code Specification) encoding: It is an extension of GB2312 encoding and is fully compatible with GB2312-80 standard. GBK encoding still uses a double-byte encoding scheme, with an encoding range of 8140-FEFE, excluding the xx7F code position, a total of 23940 code positions, and a total of 21886 Chinese characters and graphic symbols, including 21003 Chinese characters and 883 graphic symbols.

[0029] ISO-8859-1 encoding: ISO-8859-1 encoding is a single-byte encoding, which is backward compatible with ASCII. Its encoding range is 0x00-0xFF. 0x00-0xFF is completely consistent with ASCII. 0x80-0x9F is control characters, and 0xA0-0xFF is text symbols.

[0030] Example 1

[0031] According to an embodiment of the present application, a file encoding format detection method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0032] Figure 1 is a flow chart of an optional file encoding format detection method according to an embodiment of the present application, such as Figure 1 As shown, the method at least includes steps S102-S106, wherein:

[0033] Step S102, obtaining a target encoding file, and detecting whether the bytes of the target encoding file include encoding format information.

[0034] The target encoding file can be either a file or a byte stream. The encoding formats can be divided into the following three types: can identify standard encoding formats, such as Unicode encoding format, Unicode big endian encoding format, and UTF-8 encoding format; can identify non-standard encoding formats, such as UTF-16 encoding format, GBK encoding format, GB2312 encoding format, and ISO-8859-1 encoding format; can identify garbled files.

[0035] As an optional scheme of an embodiment of the present application, when detecting whether the bytes of the target encoded file include encoding format information, it can be detected whether the bytes of the target encoded file meet the following conditions: the first two bytes of the target encoded file are FF-FE, or the first two bytes of the target encoded file are FE-FF, or the first three bytes of the target encoded file are EF-BB-BF; when the bytes of the target encoded file meet any of the above conditions, it is determined that the bytes of the target encoded file include encoding format information, and the encoding format information is the first two bytes or the first three bytes of the target encoded file; when the bytes of the target encoded file do not meet all of the above conditions, it is determined that the bytes of the target encoded file do not include encoding format information.

[0036] Step S104: when the bytes of the target coded file include coding format information, determine the target coding format of the target coded file based on the coding format information.

[0037] Optionally, the target encoding format includes at least one of the following: Unicode encoding format, Unicode bigendian encoding format, and UTF-8 encoding format, wherein, compared with Unicode encoding, Unicode big endian encoding is just different in the storage order of two bytes of characters.

[0038] Specifically, when the target encoded file includes encoding format information in its bytes, the encoding format of the target encoded file can be determined based on the Unicode encoding rules and the encoding format information of the target encoded file. The specific determination rules are as follows:

[0039] When the first two bytes of the target encoding file are FF-FE, the target encoding format of the target encoding file is determined to be the Unicode encoding format; when the first two bytes of the target encoding file are FE-FF, the target encoding format of the target encoding file is determined to be the Unicode big endian encoding format; when the first three bytes of the target encoding file are EF-BB-BF, the target encoding format of the target encoding file is determined to be the UTF-8 encoding format.

[0040] Step S106, when the bytes of the target encoded file do not include encoding format information, split the target encoded file into a first preset number of target sub-files, use an encoding format detector to detect the first encoding format of each target sub-file, obtain an encoding format set, and determine the target encoding format of the target encoded file from the encoding format set.

[0041] Optionally, the target encoding format includes at least one of the following: UTF-16 encoding format, GBK encoding format, GB2312 encoding format, ISO-8859-1 encoding format.

[0042] The encoding format detector may be an open source cpdetector detector, which can detect the current mainstream encoding formats. Since the cpdetector detector uses various encoding formats to detect the target encoding files one by one, once a certain encoding format returns correct during the detection process, the detection is immediately stopped, and the format of the target encoding file is determined as the final result. When there are data of multiple encoding formats in the target encoding file, it will lead to misjudgment and obtain an erroneous recognition result.

[0043] In order to solve the above problems, the use of the cpdetector detector is improved. Specifically, the target encoding format of the target encoding file can be determined by the following methods:

[0044] The obtained target coding file is divided into n equal parts, where n is less than or equal to the length of the target coding file; for each sub-target coding file, a cpdetector detector is used to detect each sub-target coding file to obtain a coding format set, count the types of all first coding formats in the coding format set, and determine the first number of first coding formats of each type; sort the first coding formats of each type in descending order based on the first number; determine the target coding format of the target coding file based on the first two types of first coding formats sorted.

[0045] The value of n can be set according to actual conditions. The larger the value of n is, the higher the accuracy of the resulting encoding format is.

[0046] As an optional scheme of an embodiment of the present application, when the first encoding format ranked first is not garbled, the target encoding format of the target encoding file is determined to be the first encoding format ranked first; when the first encoding format ranked first is garbled, the first number of the first encoding format ranked first is compared with a preset threshold; when the first number is greater than the preset threshold, the target encoding format of the target encoding file is determined to be garbled; when the first number is not greater than the preset threshold, the target encoding format of the target encoding file is determined to be the first encoding format ranked second.

[0047] For example, when the cpdetector detector detects the target encoded file, the detection results show that the GB2312 encoding format appears 61 times and the GBK encoding format appears 42 times, and no garbled format appears. It can be determined that the target encoding format is the GB2312 encoding format.

[0048] When the cpdetector detector detects the target encoded file, the detection results show that the UTF-16 encoding format appears 14 times and the garbled format appears 24 times. In order to accurately determine the encoding format information of the target encoded file, the number of occurrences of the garbled format can be compared with the preset threshold. If the preset threshold is set to 18, the target encoding format of the target encoded file is determined to be garbled; if the preset threshold is set to 30, the target encoding format of the target encoded file is determined to be UTF-16 encoding.

[0049] The above method can be implemented based on Java code. In actual application, the Java code in the embodiment of the present application can be packaged into a jar package, and the jar package can be introduced to detect the target encoded file in the scenario where it is needed. If the bytes of the target encoded file include encoding format information, the jar package can be used to directly determine the first two or three bytes to output the target encoding format; if the bytes of the target encoded file do not include encoding format information, the improved cpdetector detector is used to identify the target encoding format of the target encoded file.

[0050] In the embodiment of the present application, the target encoding file is first obtained, and whether the bytes of the target encoding file include encoding format information is detected. When the bytes of the target encoding file include encoding format information, the target encoding format of the target encoding file is determined based on the encoding format information; when the bytes of the target encoding file do not include encoding format information, the target encoding file is split into a first preset number of target sub-files, and the first encoding format of each target sub-file is detected by using an encoding format detector to obtain an encoding format set, and the target encoding format of the target encoding file is determined from the encoding format set. In this way, it is possible to classify and detect various types of files according to whether the bytes of the target encoding file include encoding format information, thereby solving the technical problem that the relevant detection technology cannot quickly and accurately identify the encoding formats of different files.

[0051] Example 2

[0052] According to an embodiment of the present application, a file encoding format detection device for implementing the above-mentioned file encoding format detection method is also provided. Figure 2 As shown, the device at least includes a detection module 21, a first determination module 22 and a second determination module 23, wherein:

[0053] The detection module 21 is used to obtain the target encoding file and detect whether the bytes of the target encoding file include encoding format information.

[0054] The target encoding file can be either a file or a byte stream. The encoding formats can be divided into the following three types: can identify standard encoding formats, such as Unicode encoding format, Unicode big endian encoding format, and UTF-8 encoding format; can identify non-standard encoding formats, such as UTF-16 encoding format, GBK encoding format, GB2312 encoding format, and ISO-8859-1 encoding format; can identify garbled files.

[0055] As an optional scheme of an embodiment of the present application, when detecting whether the bytes of the target encoded file include encoding format information, it can be detected whether the bytes of the target encoded file meet the following conditions: the first two bytes of the target encoded file are FF-FE, or the first two bytes of the target encoded file are FE-FF, or the first three bytes of the target encoded file are EF-BB-BF; when the bytes of the target encoded file meet any of the above conditions, it is determined that the bytes of the target encoded file include encoding format information, and the encoding format information is the first two bytes or the first three bytes of the target encoded file; when the bytes of the target encoded file do not meet all of the above conditions, it is determined that the bytes of the target encoded file do not include encoding format information.

[0056] The first determination module 22 is configured to determine a target encoding format of the target encoding file based on the encoding format information when the encoding format information is included in the bytes of the target encoding file.

[0057] Optionally, the target encoding format includes at least one of the following: Unicode encoding format, Unicode bigendian encoding format, and UTF-8 encoding format, wherein, compared with Unicode encoding, Unicode big endian encoding is just different in the storage order of two bytes of characters.

[0058] Specifically, when the target encoded file includes encoding format information in its bytes, the encoding format of the target encoded file can be determined based on the Unicode encoding rules and the encoding format information of the target encoded file. The specific determination rules are as follows:

[0059] When the first two bytes of the target encoding file are FF-FE, the target encoding format of the target encoding file is determined to be the Unicode encoding format; when the first two bytes of the target encoding file are FE-FF, the target encoding format of the target encoding file is determined to be the Unicode big endian encoding format; when the first three bytes of the target encoding file are EF-BB-BF, the target encoding format of the target encoding file is determined to be the UTF-8 encoding format.

[0060] The second determination module 23 is used to split the target encoded file into a first preset number of target sub-files when the bytes of the target encoded file do not include encoding format information, use an encoding format detector to detect the first encoding format of each target sub-file, obtain an encoding format set, and determine the target encoding format of the target encoded file from the encoding format set.

[0061] Optionally, the target encoding format includes at least one of the following: UTF-16 encoding format, GBK encoding format, GB2312 encoding format, ISO-8859-1 encoding format.

[0062] The encoding format detector may be an open source cpdetector detector, which can detect the current mainstream encoding formats. Since the cpdetector detector uses various encoding formats to detect the target encoding files one by one, once a certain encoding format returns correct during the detection process, the detection is immediately stopped, and the format of the target encoding file is determined as the final result. When there are data of multiple encoding formats in the target encoding file, it will lead to misjudgment and obtain an erroneous recognition result.

[0063] In order to solve the above problems, the use of the cpdetector detector is improved. Specifically, the target encoding format of the target encoding file can be determined by the following methods:

[0064] The obtained target coding file is divided into n equal parts, where n is less than or equal to the length of the target coding file; for each sub-target coding file, a cpdetector detector is used to detect each sub-target coding file to obtain a coding format set, count the types of all first coding formats in the coding format set, and determine the first number of first coding formats of each type; sort the first coding formats of each type in descending order based on the first number; determine the target coding format of the target coding file based on the first two types of first coding formats sorted.

[0065] The value of n can be set according to actual conditions. The larger the value of n is, the higher the accuracy of the resulting encoding format is.

[0066] As an optional scheme of the embodiment of the present application, when the first encoding format ranked first is not garbled, the target encoding format of the target encoding file is determined to be the first encoding format ranked first; when the first encoding format ranked first is garbled, the first number of the first encoding format ranked first is compared with a preset threshold; when the first number is greater than the preset threshold, the target encoding format of the target encoding file is determined to be garbled; when the first number is not greater than the preset threshold, the target encoding format of the target encoding file is determined to be the first encoding format ranked second. It should be noted that each module in the file encoding format detection device in the embodiment of the present application corresponds to each implementation step of the file encoding format detection method in Example 1. Since a detailed description has been made in Example 1, some details not reflected in this embodiment can refer to Example 1, and will not be repeated here.

[0067] Example 3

[0068] According to an embodiment of the present application, a non-volatile storage medium is also provided, which includes a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the file encoding format detection method in Example 1.

[0069] According to an embodiment of the present application, a processor is also provided, which is used to run a program, wherein the file encoding format detection method in Example 1 is executed when the program is running.

[0070] According to an embodiment of the present application, an electronic device is also provided, which includes: a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to execute the file encoding format detection method in Example 1 through the computer program.

[0071] Optionally, when the program is running, the following steps are executed: obtaining a target encoding file, detecting whether the bytes of the target encoding file include encoding format information; when the bytes of the target encoding file include encoding format information, determining the target encoding format of the target encoding file based on the encoding format information; when the bytes of the target encoding file do not include encoding format information, splitting the target encoding file into a first preset number of target sub-files, using an encoding format detector to detect a first encoding format of each target sub-file, obtaining an encoding format set, and determining the target encoding format of the target encoding file from the encoding format set.

[0072] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0073] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0074] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0075] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed over multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0076] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0077] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc. Various media that can store program codes.

[0078] The above are only preferred implementations of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A file encoding format detection method, characterized in that: include: Obtaining a target encoding file, and detecting whether the bytes of the target encoding file include encoding format information; When the encoding format information is included in the bytes of the target encoding file, determining the target encoding format of the target encoding file based on the encoding format information; When the target coded file does not include the coding format information in bytes, splitting the target coded file into a first preset number of target sub-files, detecting a first coding format of each of the target sub-files using a coding format detector to obtain a coding format set, and determining the target coding format of the target coded file from the coding format set; Among them, determining the target encoding format of the target encoding file from the encoding format set includes: counting the types of all the first encoding formats in the encoding format set, and determining a first number of the first encoding formats of each type; sorting the first encoding formats of each type in descending order based on the first number; and determining the target encoding format of the target encoding file based on the first two types of the first encoding formats sorted.

2. The method according to claim 1, characterized in that: The target encoding format includes at least one of the following: Unicode encoding format, Unicode big endian encoding format, UTF-8 encoding format, UTF-16 encoding format, GBK encoding format, GB2312 encoding format, ISO-8859-1 encoding format.

3. The method according to claim 1, characterized in that Detecting whether the bytes of the target encoded file include encoding format information includes: Detect whether the bytes of the target encoding file meet the following conditions: the first two bytes of the target encoding file are FF-FE, or the first two bytes of the target encoding file are FE-FF, or the first three bytes of the target encoding file are EF-BB-BF; When the bytes of the target coded file meet any of the above conditions, determining that the bytes of the target coded file include the encoding format information, the encoding format information is the first two bytes or the first three bytes of the target coded file; When the bytes of the target coded file do not satisfy all of the above conditions, it is determined that the bytes of the target coded file do not include the coding format information.

4. The method according to claim 3, characterized in that Determining a target encoding format of the target encoding file based on the encoding format information includes: When the first two bytes of the target encoding file are FF-FE, determining that the target encoding format of the target encoding file is a Unicode encoding format; When the first two bytes of the target encoding file are FE-FF, determining that the target encoding format of the target encoding file is a Unicode big endian encoding format; When the first three bytes of the target encoding file are EF-BB-BF, it is determined that the target encoding format of the target encoding file is the UTF-8 encoding format.

5. The method according to claim 1, characterized in that: Determining the target encoding format of the target encoding file based on the first two types of the first encoding formats in the sorting order includes: When the first encoding format ranked first is not a garbled code, determining that the target encoding format of the target encoding file is the first encoding format ranked first; When the first encoding format ranked first is a garbled code, comparing the first number of the first encoding format ranked first with a preset threshold; When the first number is greater than the preset threshold, it is determined that the target encoding format of the target encoding file is garbled; when the first number is not greater than the preset threshold, it is determined that the target encoding format of the target encoding file is the second-ranked first encoding format.

6. The method according to any one of claims 1 to 5, characterized in that The encoding format detector is an open source cpdetector detector.

7. A file encoding format detection device, characterized in that: include: A detection module, used for acquiring a target encoding file and detecting whether the bytes of the target encoding file include encoding format information; A first determination module, configured to determine a target encoding format of the target encoding file based on the encoding format information when the encoding format information is included in the bytes of the target encoding file; The second determination module is used to split the target encoding file into a first preset number of target sub-files when the encoding format information is not included in the bytes of the target encoding file, use an encoding format detector to detect the first encoding format of each of the target sub-files, obtain an encoding format set, and determine the target encoding format of the target encoding file from the encoding format set; wherein, determining the target encoding format of the target encoding file from the encoding format set includes: counting the types of all the first encoding formats in the encoding format set, and determining a first number of the first encoding formats of each type; sorting the first encoding formats of each type in descending order based on the first number; and determining the target encoding format of the target encoding file based on the first two types of the first encoding formats sorted first.

8. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the file encoding format detection method according to any one of claims 1 to 6.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the file encoding format detection method according to any one of claims 1 to 6 through the computer program.

Images