A text type recognition method, device, electronic device and storage medium

By combining the file suffix name and keyword score in the text content, the misjudgment problem caused by tampering with suffix name in the prior art is solved, the accuracy of text type recognition is improved, and more reliable language type detection is achieved.

CN113111147BActive Publication Date: 2024-12-03SANGFOR TECH INC
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
CN202010032858.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-01-13
Publication Date
2024-12-03
Estimated Expiration
2040-01-13

AI Technical Summary

Technical Problem

When detecting file language types, the prior art relies on file suffix names for recognition, which is prone to misjudgment caused by attackers tampering with suffix names, and lacks effective text type recognition methods to improve recognition accuracy.

Method used

By obtaining the suffix name of the file to be detected and the keywords in the text content, combining the multi-mode matching algorithm to extract keywords, calculate the score of each language type, and judge the file type based on the score and threshold, and determine the final result of the suffix name and keyword recognition results. type.

Benefits of technology

It improves the accuracy of text type recognition, avoids misjudgment problems when relying solely on suffix names to identify, and enhances the reliability of detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113111147B_ABST
    Figure CN113111147B_ABST
Patent Text Reader

Abstract

The present application discloses a text type recognition method, device, electronic device and computer-readable storage medium, the method comprising: obtaining a file to be detected; identifying the suffix of the file to be detected to obtain a first recognition result; identifying keywords in the text content of the file to be detected to obtain a second recognition result; combining the first recognition result and the second recognition result to determine the text type of the file to be detected. As can be seen from the above, when performing text type recognition, the present application not only recognizes the suffix of the file to be detected, but also recognizes the keywords in the text content of the file, and obtains the text type of the file to be detected by combining the recognition result of the suffix and the recognition result of the keyword, thereby avoiding the misjudgment problem that may be caused by only identifying the suffix, and improving the recognition accuracy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more specifically, to a text type recognition method and device, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the development of the Internet, more and more language categories have emerged, and the styles of these languages ​​are similar or completely different. When a detection engine needs to process multiple different languages ​​at the same time, such as malicious Webshell backdoor detection, if the program detection is performed without distinguishing the language type, false positives will be generated, because the attack methods of two script languages ​​may be completely opposite. Therefore, a text type recognition method is needed to first identify the language type of the file to be detected, and then detect it according to the characteristics of each language.

[0003] When traditional technologies identify the language type of a file to be detected, they usually directly determine the language type of the file by the file's suffix. However, this method is based on the premise of trusting the file suffix. If in some scenarios, the attacker changes the script suffix to another language, it will lead to misjudgment of the text type. Therefore, how to provide a text type identification method that solves the above problem is what technicians in this field need to focus on. Summary of the invention

[0004] The purpose of the present application is to provide a text type recognition method, device, electronic device and computer-readable storage medium to improve recognition accuracy.

[0005] To achieve the above objectives, the present application provides a text type recognition method, comprising:

[0006] Get the file to be tested;

[0007] Identify the suffix of the file to be detected to obtain a first identification result;

[0008] Recognize keywords in the text content of the file to be detected to obtain a second recognition result;

[0009] The text type of the to-be-detected file is determined by combining the first recognition result and the second recognition result.

[0010] Optionally, identifying the suffix of the file to be detected to obtain a first identification result includes:

[0011] Obtain the file name of the file to be detected;

[0012] Extracting the suffix of the file to be detected from the file name;

[0013] A search is performed in a preset character string set to determine whether there is a character string identical to the suffix name, and a first recognition result for the suffix name is obtained.

[0014] Optionally, identifying keywords in the text content of the file to be detected to obtain a second identification result includes:

[0015] Reading the text content of the file to be detected, and extracting keywords from the text content by using multi-mode matching;

[0016] Analyze and count each keyword to get the score corresponding to each language type;

[0017] The language type with the highest score is determined as the second recognition result.

[0018] Optionally, also include:

[0019] Collect keywords corresponding to each language type to obtain a preset keyword set;

[0020] By statistically analyzing the occurrence probability of each preset keyword in the preset keyword set, or performing weighted calculation on each preset keyword using machine learning, the preset weight corresponding to each preset keyword in each language type is obtained.

[0021] Optionally, analyze and count each keyword to obtain the score corresponding to each language type, including:

[0022] Count the number of occurrences of each keyword and obtain the preset weight corresponding to each keyword in each language type;

[0023] A weighted score corresponding to each language type is calculated according to the number of occurrences and the preset weight.

[0024] Optionally, combining the first recognition result and the second recognition result to determine the text type of the to-be-detected file includes:

[0025] If the first recognition result and the second recognition result are the same, directly determining the first recognition result or the second recognition result as the text type of the file to be detected;

[0026] If the first recognition result and the second recognition result are different, determining whether the weighted score is greater than or equal to a preset threshold;

[0027] If yes, determining the second recognition result as the text type of the file to be detected;

[0028] If not, the first recognition result is determined as the text type of the file to be detected.

[0029] To achieve the above object, the present application provides a text type recognition device, comprising:

[0030] A file acquisition module, used to acquire files to be detected;

[0031] A first recognition module, used to recognize the suffix of the file to be detected and obtain a first recognition result;

[0032] A second recognition module is used to recognize keywords in the text content of the file to be detected to obtain a second recognition result;

[0033] A type determination module is used to determine the text type of the file to be detected by combining the first recognition result and the second recognition result.

[0034] Optionally, the second identification module includes:

[0035] A keyword extraction unit, used for reading the text content of the file to be detected, and extracting keywords from the text content by using multi-mode matching;

[0036] The score statistics unit is used to analyze and count each keyword to obtain the score corresponding to each language type;

[0037] A result determination unit is used to determine the language type with the highest score as the second recognition result.

[0038] To achieve the above objectives, the present application provides an electronic device, including:

[0039] Memory for storing computer programs;

[0040] A processor is used to implement the steps of any of the above-disclosed text type recognition methods when executing the computer program.

[0041] To achieve the above objectives, the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-disclosed text type recognition methods are implemented.

[0042] Through the above scheme, it can be seen that a text type identification method provided by the present application includes: obtaining a file to be detected; identifying the suffix name of the file to be detected to obtain a first identification result; identifying keywords in the text content of the file to be detected to obtain a second identification result; combining the first identification result and the second identification result to determine the text type of the file to be detected. From the above, it can be seen that when performing text type identification, the present application not only identifies the suffix name of the file to be detected, but also identifies the keywords in the text content of the file, and obtains the text type of the file to be detected by combining the identification results of the suffix name and the identification results of the keywords, thereby avoiding the misjudgment problem that may be caused by identification only by the suffix name, and improving the recognition accuracy.

[0043] The present application also discloses a text type recognition device, an electronic device and a computer-readable storage medium, which can also achieve the above-mentioned technical effects.

[0044] It should be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 A flowchart of a text type recognition method disclosed in an embodiment of the present application;

[0047] Figure 2 A flowchart of another text type recognition method disclosed in an embodiment of the present application;

[0048] Figure 3 A structural diagram of a text type recognition device disclosed in an embodiment of the present application;

[0049] Figure 4 A structural diagram of an electronic device disclosed in an embodiment of the present application;

[0050] Figure 5 This is a structural diagram of another electronic device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0052] In the prior art, when identifying the language type of a file to be detected, the language type of the file is usually determined directly by the file suffix. However, this method is based on the premise of trusting the file suffix. If in some scenarios, the attacker changes the script suffix to another language, it will lead to misjudgment of the text type.

[0053] Therefore, an embodiment of the present application discloses a text type recognition method to improve recognition accuracy.

[0054] See also Figure 1 As shown, a text type recognition method disclosed in an embodiment of the present application includes:

[0055] S101: Obtain the file to be detected;

[0056] In the embodiment of the present application, it is first necessary to obtain a file to be detected. The file to be detected is a file whose text type needs to be detected, which can be a script file, etc. When obtaining the file to be detected, a preset receiving interface can be provided to obtain the file to be detected sent by the user, and the file to be detected uploaded by the user can also be obtained through a preset import interface, and the file to be detected can also be downloaded through the network. The present application does not specifically limit the process of obtaining the file to be detected.

[0057] S102: Identify the suffix of the file to be detected to obtain a first identification result;

[0058] In this step, the first recognition result for the suffix name is obtained by identifying the suffix name of the file to be detected. Specifically, the process of identifying the suffix name of the file to be detected and obtaining the first recognition result may include: obtaining the file name of the file to be detected; extracting the suffix name of the file to be detected from the file name; and searching in a preset character string set whether there is a character string identical to the suffix name, and obtaining the first recognition result for the suffix name.

[0059] It is understandable that the file suffix is ​​usually added to the end of the main file name, and the suffix and the main file name are separated by the "." separator. The file name includes the main file name, the suffix and the separator. After obtaining the file name of the file to be detected, the file suffix after the separator can be extracted by identifying the separator.

[0060] It should be noted that the present application can collect all possible file types in advance, and use all suffixes corresponding to all possible file types as a preset string set. After extracting the file suffix, the suffix can be used to match the preset string set. If the match is successful, it indicates that the suffix of the current file has been identified, and the file type corresponding to the suffix is ​​the identification result for the suffix.

[0061] S103: Recognize keywords in the text content of the file to be detected to obtain a second recognition result;

[0062] In the embodiment of the present application, the text content of the file to be detected is read, and the keywords in the text content are identified to obtain a second recognition result for the keywords.

[0063] It should be pointed out that the embodiment of the present application does not limit the execution order between the above-mentioned step S102 and step S103, that is, step S102 can be executed first, or step S103 can be executed first, or both steps can be executed at the same time to achieve concurrent recognition processing.

[0064] S104: Determine the text type of the to-be-detected file by combining the first recognition result and the second recognition result.

[0065] It is understandable that after obtaining the first recognition result for the suffix name and the second recognition result for the keyword, the present application can obtain a more accurate text type recognition result by comprehensively analyzing the two results.

[0066] In a specific implementation, if the corresponding recognition result cannot be obtained after the suffix name is recognized, the second recognition result for the keyword can be directly determined as the text type of the file to be detected. If the corresponding recognition result cannot be obtained for both the suffix name and the keyword, it can be determined that the recognition of the current file to be detected has failed, and a prompt message can be output to remind the user to perform manual recognition.

[0067] Through the above scheme, it can be seen that a text type identification method provided by the present application includes: obtaining a file to be detected; identifying the suffix name of the file to be detected to obtain a first identification result; identifying keywords in the text content of the file to be detected to obtain a second identification result; combining the first identification result and the second identification result to determine the text type of the file to be detected. From the above, it can be seen that when performing text type identification, the present application not only identifies the suffix name of the file to be detected, but also identifies the keywords in the text content of the file, and obtains the text type of the file to be detected by combining the identification results of the suffix name and the identification results of the keywords, thereby avoiding the misjudgment problem that may be caused by identification only by the suffix name, and improving the recognition accuracy.

[0068] The present application embodiment discloses another text type recognition method. Compared with the previous embodiment, this embodiment further explains and optimizes the technical solution. Figure 2 As shown, specifically:

[0069] S201: Obtain the file to be detected;

[0070] S202: Identify the suffix of the file to be detected to obtain a first identification result;

[0071] S203: reading the text content of the file to be detected, and extracting keywords from the text content by using multi-mode matching;

[0072] In the embodiment of the present application, the text content of the file to be detected is read, and then a multi-mode matching algorithm is used to extract keywords from the text content. The multi-mode matching algorithm is specifically an algorithm for matching multiple keywords from a text. The AC (Aho-Corasick automaton) automaton is a common multi-mode matching algorithm. The process of multi-mode matching using the AC automaton can specifically include: first, a dictionary tree is constructed using all preset keywords, and a fail pointer can be constructed on the dictionary tree. The fail pointer is used to determine the position from which the matching should continue after the matching fails on the dictionary tree. Then, the text content can be used to match keywords on the dictionary tree.

[0073] S204: Analyze and count each keyword to obtain a score corresponding to each language type;

[0074] In this step, after the keywords are extracted, the number of occurrences of each keyword can be counted, and the preset weight corresponding to each keyword in each language type can be obtained, and the weighted score corresponding to each language type can be calculated based on the number of occurrences and the preset weight. It is understandable that the same keyword may exist in multiple languages, but the weight represented in each language may be different, so the same keyword may correspond to multiple weights, that is, different weights in multiple language types.

[0075] It should be noted that the embodiments of the present application may collect keywords corresponding to each language type in advance to obtain a preset keyword set, and weight each preset keyword in the preset keyword set to determine the preset weight corresponding to each preset keyword under each language type. When weighting each preset keyword in the preset keyword set, the occurrence probability of each preset keyword in the preset keyword set may be statistically analyzed to obtain the preset weight, or machine learning may be used to perform weighted calculation on each preset keyword to obtain the preset weight. As a specific implementation method, the present application may first collect corresponding sample data for each language type as a sample set for machine learning training, and then pre-process the sample set, that is, process each sample to generate a corresponding feature string, the feature may be a keyword, each keyword is a one-dimensional feature, and the feature string is a sequence composed of the number of times each keyword appears. When training a machine learning model, firstly, a machine learning algorithm may be selected, and the generated feature string may be put into the algorithm for training, and the corresponding machine learning model may be obtained after the training is completed. The machine learning model can also be used to predict the type of script. The machine learning model also generates feature importance scores, i.e., the weights of keywords. Feature importance scores can be output in some common algorithms, such as XGBoost, GDBT, Random Forest, and tree. In the process of using machine learning to train the model, the importance index of each feature can be obtained by calling the feature_importance_() function, and then this importance index can be used as the weight of the keyword.

[0076] After counting the number of occurrences of each keyword and obtaining the preset weights corresponding to each keyword in each language type, the weighted score corresponding to each language type can be calculated using the preset score calculation formula. The preset score calculation formula is: Among them, num() is the number of times the keyword appears, weight() is the preset weight of the keyword, all_word is all keywords under the current language type, and Score is the calculated weighted score.

[0077] S205: Determine the language type with the highest score as the second recognition result;

[0078] It is understandable that, in the embodiment of the present application, after obtaining the score corresponding to each language type, the language type with the highest score may be determined as the second recognition result for the keyword.

[0079] S206: Determine the text type of the file to be detected by combining the first recognition result and the second recognition result.

[0080] In an embodiment of the present application, if the above-mentioned first recognition result and the second recognition result are the same, the text type corresponding to the first recognition result or the second recognition result can be directly determined as the text type of the file to be detected; if the first recognition result and the second recognition result are not the same, it is determined whether the weighted score obtained in the second recognition result is greater than or equal to the preset threshold; if the weighted score is greater than or equal to the preset threshold, the text type corresponding to the second recognition result can be determined as the text type of the file to be detected; if the weighted score is less than the preset threshold, the text type corresponding to the first recognition result can be determined as the text type of the file to be detected. The above-mentioned preset threshold is a threshold that can be set according to the specific scenario during the specific implementation process, and it is not limited here. If the weighted score is less than the preset threshold, it indicates that the accuracy of the corresponding recognition result is low, and there may be a misjudgment. Therefore, the recognition result for the suffix name can be used as the final recognition result; if the weighted score is greater than or equal to the preset threshold, it indicates that the accuracy of the corresponding recognition result is high, and the recognition result for the keyword can be directly used as the final recognition result.

[0081] A text type recognition device provided in an embodiment of the present application is introduced below. The text type recognition device described below and the text type recognition method described above can be referenced to each other.

[0082] See also Figure 3 As shown, a text type recognition device provided in an embodiment of the present application includes:

[0083] The file acquisition module 301 is used to acquire the file to be detected;

[0084] A first recognition module 302 is used to recognize the suffix of the file to be detected and obtain a first recognition result;

[0085] The second recognition module 303 is used to recognize keywords in the text content of the file to be detected to obtain a second recognition result;

[0086] The type determination module 304 is used to determine the text type of the to-be-detected file by combining the first recognition result and the second recognition result.

[0087] Based on the above embodiment, as a preferred implementation, the first identification module 302 includes:

[0088] A name acquisition unit, used to acquire the file name of the file to be detected;

[0089] A suffix unit, used for extracting the suffix of the file to be detected from the file name;

[0090] The character string search unit is used to search whether there is a character string identical to the suffix name in a preset character string set, and obtain a first recognition result for the suffix name.

[0091] Based on the above embodiment, as a preferred implementation, the second identification module 303 includes:

[0092] A keyword extraction unit, used for reading the text content of the file to be detected, and extracting keywords from the text content by using multi-mode matching;

[0093] The score statistics unit is used to analyze and count each keyword to obtain the score corresponding to each language type;

[0094] A result determination unit is used to determine the language type with the highest score as the second recognition result.

[0095] Based on the above embodiment, as a preferred implementation, the text type recognition device may further include:

[0096] A keyword collection module is used to collect keywords corresponding to each language type to obtain a preset keyword set;

[0097] The weighted calculation module is used to obtain the preset weight corresponding to each preset keyword in each language type by statistically analyzing the occurrence probability of each preset keyword in the preset keyword set or performing weighted calculation on each preset keyword using machine learning.

[0098] Based on the above embodiment, as a preferred implementation, the score statistics unit includes:

[0099] The weight acquisition subunit is used to count the number of occurrences of each keyword and obtain the preset weight corresponding to each keyword in each language type;

[0100] The score calculation subunit is used to calculate the weighted score corresponding to each language type according to the number of occurrences and the preset weight.

[0101] Based on the above embodiment, as a preferred implementation, the type determination module 304 includes:

[0102] a first determining unit, configured to directly determine the first recognition result or the second recognition result as the text type of the file to be detected if the first recognition result and the second recognition result are the same;

[0103] A score determination module, configured to determine whether the weighted score is greater than or equal to a preset threshold if the first recognition result and the second recognition result are different;

[0104] A second determination unit, configured to determine the second recognition result as the text type of the file to be detected if the weighted score is greater than or equal to a preset threshold;

[0105] The third determining unit is configured to determine the first recognition result as the text type of the file to be detected if the weighted score is less than a preset threshold.

[0106] The present application also provides an electronic device, see Figure 4 As shown, an electronic device provided in an embodiment of the present application includes:

[0107] A memory 100, used for storing computer programs;

[0108] The processor 200 can implement the steps provided in the above embodiment when being used to execute the computer program.

[0109] Specifically, the memory 100 includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer-readable instructions, and the internal memory provides an environment for the operation of the operating system and computer-readable instructions in the non-volatile storage medium. In some embodiments, the processor 200 can be a central processing unit (CPU), a controller, a microcontroller, a microprocessor or other data processing chip, which provides computing and control capabilities for electronic devices. When executing the computer program stored in the memory 100, the steps of the text type recognition method disclosed in any of the aforementioned embodiments can be implemented.

[0110] Based on the above embodiments, as a preferred embodiment, see Figure 5 As shown, the electronic device also includes:

[0111] The input interface 300 is connected to the processor 200 and is used to obtain externally imported computer programs, parameters and instructions, and save them in the memory 100 under the control of the processor 200. The input interface 300 can be connected to an input device to receive parameters or instructions manually input by a user. The input device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the terminal housing, or a keyboard, touchpad or mouse.

[0112] The display unit 400 is connected to the processor 200 and is used to display data processed by the processor 200 and to display a visual user interface. The display unit 400 can be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc.

[0113] The network port 500 is connected to the processor 200 and is used to communicate with external terminal devices. The communication technology used for the communication connection can be a wired communication technology or a wireless communication technology, such as mobile high-definition link technology (MHL), universal serial bus (USB), high-definition multimedia interface (HDMI), wireless fidelity technology (WiFi), Bluetooth communication technology, low-power Bluetooth communication technology, communication technology based on IEEE802.11s, etc.

[0114] Figure 5 Only an electronic device having components 100-500 is shown, and those skilled in the art will appreciate that Figure 5 The structure shown does not constitute a limitation on the electronic device, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0115] The present application also provides a computer-readable storage medium, which may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes. The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the text type recognition method disclosed in any of the aforementioned embodiments are implemented.

[0116] When performing text type recognition, the present application not only recognizes the suffix name of the file to be detected, but also recognizes the keywords in the text content of the file, and obtains the text type of the file to be detected by combining the recognition result of the suffix name and the recognition result of the keyword, thereby avoiding the misjudgment problem that may be caused by recognition only by the suffix name, and improving the recognition accuracy.

[0117] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of this application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of this application.

[0118] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. A text type identification method, It is characterized in that include: Get the file to be tested; Identify the suffix of the file to be detected to obtain a first identification result; Recognize the keywords in the text content of the file to be detected to obtain a second recognition result; wherein, the recognition of the suffix name and the recognition of the keywords are performed in no particular order; Determine the text type of the to-be-detected file by combining the first recognition result and the second recognition result; The step of identifying keywords in the text content of the file to be detected to obtain a second identification result includes: Reading the text content of the file to be detected, and extracting keywords from the text content by using multi-mode matching; Analyze and count each keyword to get the score corresponding to each language type; Determine the language type with the highest score as the second recognition result; The combining of the first recognition result and the second recognition result to determine the text type of the to-be-detected file includes: If the first recognition result is not obtained after identifying the suffix name of the file to be detected, the second recognition result is directly determined as the text type of the file to be detected.

2. The text type recognition method according to claim 1, It is characterized in that Identify the suffix of the file to be detected to obtain a first identification result, including: Obtain the file name of the file to be detected; Extracting the suffix of the file to be detected from the file name; A search is performed in a preset character string set to determine whether there is a character string identical to the suffix name, and a first recognition result for the suffix name is obtained.

3. The text type recognition method according to claim 1, It is characterized in that Also includes: Collect keywords corresponding to each language type to obtain a preset keyword set; By statistically analyzing the occurrence probability of each preset keyword in the preset keyword set, or performing weighted calculation on each preset keyword using machine learning, the preset weight corresponding to each preset keyword in each language type is obtained.

4. The text type recognition method according to claim 3, It is characterized in that Analyze and count each keyword to get the score corresponding to each language type, including: Count the number of occurrences of each keyword and obtain the preset weight corresponding to each keyword in each language type; A weighted score corresponding to each language type is calculated according to the number of occurrences and the preset weight.

5. The text type recognition method according to claim 4, It is characterized in that The combining the first recognition result and the second recognition result to determine the text type of the to-be-detected file includes: If the first recognition result and the second recognition result are the same, directly determining the first recognition result or the second recognition result as the text type of the file to be detected; If the first recognition result and the second recognition result are different, determining whether the weighted score is greater than or equal to a preset threshold; If yes, determining the second recognition result as the text type of the file to be detected; If not, the first recognition result is determined as the text type of the file to be detected.

6. A text type recognition device, It is characterized in that include: A file acquisition module, used to acquire files to be detected; A first recognition module, used to recognize the suffix of the file to be detected and obtain a first recognition result; A second recognition module is used to recognize keywords in the text content of the file to be detected to obtain a second recognition result; wherein the recognition of the suffix name and the recognition of the keyword are performed in no particular order; A type determination module, used to determine the text type of the file to be detected by combining the first recognition result and the second recognition result; Wherein, the second identification module includes: A keyword extraction unit, used for reading the text content of the file to be detected, and extracting keywords from the text content by using multi-mode matching; The score statistics unit is used to analyze and count each keyword to obtain the score corresponding to each language type; A result determination unit, configured to determine the language type with the highest score as the second recognition result; The type determination module is specifically used to: if the first recognition result is not obtained after the suffix name of the file to be detected is identified, directly determine the second recognition result as the text type of the file to be detected.

7. An electronic device, It is characterized in that include: Memory for storing computer programs; A processor, configured to implement the steps of the text type recognition method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the text type recognition method according to any one of claims 1 to 5 are implemented.

Citation Information

Patent Citations

  • File type identification method and identification system

    CN103209170A

  • File type identification method and device

    CN105808583A

  • Multistage-filtering source code data detection method and device

    CN106250769A

  • Name entity recognition method and apparatus based on artificial intelligence

    CN106503192A

  • File recognizing method, device and equipment and storage medium

    CN108460155A