A file processing method, device and computer readable storage medium

Abstract

Embodiments of the present application disclose a file processing method, which comprises: obtaining a first syntax file for a first language and a second syntax file for a second language; and / or, obtaining a first lexical file for the first language and a second lexical file for the second language; in a case where it is determined that the first language and the second language have language compatibility conditions, determining a target lexical file based on the first lexical file and the second lexical file; and / or, in a case where it is determined that the first language and the second language have language compatibility conditions, determining a target syntax file based on the first syntax file and the second syntax file; and processing a to-be-processed file based on the target lexical file and / or the target syntax file. Embodiments of the present application also disclose a file processing device and a computer readable storage medium.

Description

Technical Field

[0001] This application relates to document processing technology in the field of document processing, and more particularly to a document processing method, apparatus and computer-readable storage medium. Background Technology

[0002] With the rapid development of the internet, hackers exploit vulnerabilities in the World Wide Web (WWW) to remotely inject malicious code, thereby controlling servers or stealing server information. Processing code in unknown languages ​​on the WWW to determine its malicious nature is an effective means of preventing hackers from controlling or stealing server information. Currently, processing code in unknown languages ​​requires using syntax and lexical files from multiple languages ​​to attempt processing the code and determine its meaning. This allows for subsequent determination of whether the code is malicious. However, processing code in unknown languages ​​requires sequentially trying syntax and lexical files from multiple languages, resulting in low processing efficiency. Summary of the Invention

[0003] To address the aforementioned technical problems, embodiments of this application provide a file processing method, apparatus, and computer-readable storage medium, which solves the problem of low efficiency in processing files and improves the efficiency of file processing.

[0004] A file processing method, the method comprising:

[0005] Obtain a first grammar file for the first language and a second grammar file for the second language; and / or,

[0006] Obtain a first lexical file for the first language and a second lexical file for the second language;

[0007] If it is determined that the first language and the second language are compatible, a target lexical file is determined based on the first lexical file and the second lexical file; wherein, the target lexical file includes a file that integrates the lexical structures of the first language and the second language; and / or,

[0008] If it is determined that the first language and the second language are compatible, a target grammar file is determined based on the first grammar file and the second grammar file; wherein, the target grammar file includes a file that integrates the grammar of the first language and the grammar of the second language;

[0009] The file to be processed is processed based on the target lexical file and / or the target syntax file.

[0010] In the above scheme, determining the target lexical file based on the first lexical file and the second lexical file includes:

[0011] Obtain the first word from the first lexical file and the second word from the second lexical file;

[0012] Determine the first word information of the first word and the second word information of the second word;

[0013] The target lexical file is determined based on the first word, the second word, the information of the first word, and the information of the second word.

[0014] In the above scheme, determining the first word information of the first word and the second word information of the second word includes:

[0015] Determine the first regular expression for the first word from the first lexical file, and determine the second regular expression for the second word from the second lexical file;

[0016] Accordingly, determining the target lexical file based on the first word, the second word, the information of the first word, and the information of the second word includes:

[0017] The target lexical file is determined based on the first word, the second word, the first regular expression, and the second regular expression.

[0018] In the above scheme, determining the target lexical file based on the first word, the second word, the first regular expression, and the second regular expression includes:

[0019] Determine the matching degree between the first regular expression and the second regular expression;

[0020] When the matching degree is the target threshold, the first lexical type of the first word and the second lexical type of the second word are set to the same lexical type, and the target lexical file is determined based on the first word, the second word, and the first regular expression;

[0021] If the matching degree is greater than zero and less than the target threshold, and the first lexical type and the second lexical type are the same, or if the matching degree is zero, the target lexical file is determined based on the first word, the second word, the first regular expression, and the second regular expression.

[0022] The method in the above scheme further includes:

[0023] In cases where the matching degree is greater than zero and less than the target threshold, and the first lexical type and the second lexical type are different, the first association relationship between the first words and the second association relationship between the second words are obtained, and the target lexical file is determined based on the first words, the second words, the first association relationship, the second association relationship, the first regular expression, and the second regular expression.

[0024] The method in the above scheme further includes:

[0025] In the case where the matching degree is greater than zero and less than the target threshold, and the first lexical type and the second lexical type are different, a first lexical path is determined based on the first word and the first regular expression, and a second lexical path is determined based on the second word and the second regular expression; wherein, the target lexical file includes the first lexical path and the second lexical path.

[0026] In the above scheme, determining the target syntax file based on the first syntax file and the second syntax file includes:

[0027] Obtain the first statement from the first grammar file and the second statement from the second grammar file;

[0028] The target syntax file is determined based on the first statement and the second statement.

[0029] In the above scheme, determining the target syntax file based on the first statement and the second statement includes:

[0030] Obtain the first grammatical formula of the first statement and the second grammatical formula of the second statement;

[0031] Determine the first sentence structure relationship between the sentence structures corresponding to the first grammatical constraint, and determine the second sentence structure relationship between the sentence structures corresponding to the second grammatical constraint;

[0032] Based on the first sentence structure association relationship and the second sentence structure association relationship, the first statement and the second statement are combined, and the second grammatical constraint and the first grammatical constraint are combined to obtain the target grammar file.

[0033] In the above scheme, the step of combining the first and second statements based on the first and second sentence structure associations, and combining the second grammatical constraint with the first grammatical constraint to obtain the target grammar file, includes:

[0034] Based on the first sentence structure association relationship and the second sentence structure association relationship, the first sentence and the second sentence are combined, and the second grammar constraint is combined with the first grammar constraint to obtain the candidate grammar file;

[0035] If there are grammatical errors in the candidate grammar files, the candidate grammar files are corrected to obtain the target grammar file;

[0036] If there are no syntax errors in the candidate syntax files, the target syntax file is selected as the candidate syntax file.

[0037] In the above scheme, the step of correcting the candidate syntax file to obtain the target syntax file when there are syntax errors in the candidate syntax file includes:

[0038] The third grammar rules in the candidate grammar files are adjusted to obtain the target grammar file; or

[0039] If the candidate syntax file contains errors, delete the erroneous content in the candidate syntax file to obtain the target syntax file;

[0040] If the candidate grammar file contains errors, the length of the third grammar rule in the candidate grammar file is increased to obtain the target grammar file;

[0041] If the candidate grammar file contains errors, the grammatical relationships between the third-level grammar rules are obtained, and the target grammar file is determined based on the grammatical relationships and the third-level grammar rules.

[0042] In the above scheme, obtaining the first lexical file for the first language and the second lexical file for the second language includes:

[0043] If the first language and the second language are compatible, obtain the first lexical file and the second lexical file;

[0044] Accordingly, the method further includes:

[0045] If the first language and the second language are not compatible, an alarm message is generated and output, wherein the alarm message is used to indicate that the first language and the second language are incompatible.

[0046] In the above scheme, before determining the target lexical file based on the first and second lexical files, assuming that the first and second languages ​​are compatible, the method further includes:

[0047] The complexity of the first language is determined based on the first grammar structure in the first grammar file, and the complexity of the second language is determined based on the second grammar structure in the second grammar file.

[0048] Based on the similarity between the identifiers in the first grammar file and the identifiers in the second grammar file, the language similarity between the first grammar file and the second grammar file is determined.

[0049] Based on the complexity of the first language, the complexity of the second language, and the language similarity, it is determined whether the first language and the second language meet the language compatibility criteria.

[0050] In the above scheme, determining whether the first language and the second language are compatible based on the complexity of the first language, the complexity of the second language, and the language similarity includes:

[0051] If the complexity of the first language and the complexity of the second language are both no greater than the complexity threshold, and the language similarity is greater than the similarity threshold, then the first language and the second language are determined to have language compatibility.

[0052] If the complexity of the first language or the complexity of the second language is greater than the complexity threshold, or the language similarity is not greater than the language similarity threshold, it is determined that the first language and the second language do not meet the language compatibility conditions.

[0053] In the above scheme, the step of processing the file to be processed based on the target lexical file and / or the target syntax file includes:

[0054] Retrieve the file to be processed containing code;

[0055] Based on the target lexical file, words in the code are detected to determine a first detection result; and / or,

[0056] Based on the target syntax file, the statements in the unknown code are detected to determine the second detection result;

[0057] Based on the first detection result and / or the second detection result, determine whether the file to be processed is a malicious file.

[0058] A file processing method, the method comprising:

[0059] Obtain grammar files for at least two of the multiple languages; and / or,

[0060] Obtain lexical files for at least two of the multiple languages;

[0061] If it is determined that at least two languages ​​meet the language compatibility requirements, a target lexical file is determined based on the lexical files of the at least two languages; wherein, the target lexical file includes a file that integrates the lexical files of the at least two languages; and / or,

[0062] If it is determined that at least two languages ​​meet the language compatibility requirements, a target grammar file is determined based on the grammar files of the at least two languages; wherein, the target grammar file includes a file that integrates the grammars of the at least two languages;

[0063] The file to be processed is processed based on the target lexical file and / or the target syntax file.

[0064] A file processing method, the method comprising:

[0065] Obtain the first grammar file for the first language and the second grammar file for the second language;

[0066] The complexity of the first language is determined based on the first grammar structure in the first grammar file, and the complexity of the second language is determined based on the second grammar structure in the second grammar file.

[0067] Based on the similarity between the identifiers in the first grammar file and the identifiers in the second grammar file, the language similarity between the first grammar file and the second grammar file is determined.

[0068] Based on the complexity of the first language, the complexity of the second language, and the language similarity, it is determined whether the first language and the second language meet the language compatibility criteria.

[0069] A file processing device, the device comprising: a processor, a memory, and a communication bus;

[0070] The communication bus is used to realize the communication connection between the processor and the memory;

[0071] The processor is used to execute a file processing program in the memory to implement the steps of the file processing method described above.

[0072] A computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the steps of the file processing method described above.

[0073] The document processing method, apparatus, and computer-readable storage medium provided in this application embodiment acquire a first grammar file for a first language and a second grammar file for a second language; and / or, acquire a first lexical file for a first language and a second lexical file for a second language; when it is determined that the first language and the second language have language compatibility conditions, a target lexical file is determined based on the first lexical file and the second lexical file; wherein, the target lexical file includes a file that integrates the lexicals of the first language and the second language; and / or, when it is determined that the first language and the second language have language compatibility conditions, a target lexical file is determined based on the first grammar file and the second grammar file. The target grammar file is determined; the target grammar file includes a file that integrates the grammars of the first language and the grammars of the second language; the file to be processed is processed based on the target lexical file and / or the target grammar file; in this way, the target grammar file and / or the target lexical file that integrates the grammars of multiple languages ​​are used to process the file to be processed, which eliminates the need to try to process the file to be processed with the grammar file and lexical file of each language one by one, thus improving the efficiency of processing the file to be processed, and also eliminating the need to load multiple grammar files and multiple lexical files when processing the file to be processed, further reducing development costs. Attached Figure Description

[0074] Figure 1 A flowchart illustrating a document processing method provided in an embodiment of this application;

[0075] Figure 2 A flowchart illustrating a document processing method according to another embodiment of this application;

[0076] Figure 3 A flowchart illustrating a document processing method provided in another embodiment of this application;

[0077] Figure 4 A flowchart illustrating a document processing method provided in another embodiment of this application;

[0078] Figure 5 A flowchart illustrating a document processing method provided in another embodiment of this application;

[0079] Figure 6 This is a schematic diagram of the structure of a first document processing device provided in an embodiment of this application;

[0080] Figure 7 This is a schematic diagram of the structure of a second document processing device provided in an embodiment of this application;

[0081] Figure 8 This is a schematic diagram of the structure of a third document processing device provided in an embodiment of this application;

[0082] Figure 9 This is a schematic diagram of the structure of a file processing device provided in an embodiment of this application. Detailed Implementation

[0083] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0084] This application provides a file processing method, such as... Figure 1 As shown, the method includes the following steps:

[0085] Step 101: Obtain the first grammar file for the first language and the second grammar file for the second language.

[0086] Here, both the first language and the second language refer to programming languages; programming languages ​​may include, but are not limited to, C, C++, and Java; the first syntax file is a file that represents the syntax of the first language; the second syntax file is a file that represents the syntax of the second language.

[0087] In this embodiment, the first grammar file can be a grammar file created by the file processing device based on the implementation logic of the grammar parsing of the first language, supporting the grammar of the first language; the second grammar file can be a grammar file created by the file processing device based on the implementation logic of the grammar parsing of the second language, supporting the grammar of the second language. The first and second grammar files can also be actively downloaded by the file processing device via the network; or they can be downloaded by the file processing device upon receiving a download instruction from a user via the network, or sent to the file processing device by the user through a terminal.

[0088] In one feasible implementation, the first syntax file can be a syntax file for the C language, and the second syntax file can be a syntax file for the C++ language.

[0089] It should be noted that the first syntax file can be obtained first, followed by the second syntax file; or the second syntax file can be obtained first, followed by the first syntax file; or the first and second syntax files can be obtained simultaneously. In this embodiment, the execution order of obtaining the first and second syntax files is not limited.

[0090] Step 102: Obtain the first lexical file for the first language and the second lexical file for the second language.

[0091] The first lexical file is a file representing the lexical structure of the first language; the second lexical file is a file representing the lexical structure of the second language.

[0092] In this embodiment, the first lexical file can be a lexical file created by the file processing device based on the lexical parsing implementation logic of the first language, specifically for the lexical support of the first language; the second lexical file can be a lexical file created by the file processing device based on the lexical parsing implementation logic of the second language, specifically for the lexical support of the second language. The first and second lexical files can also be actively downloaded by the file processing device via a network; alternatively, they can be downloaded by the file processing device upon receiving a download instruction from a user via a network, or sent to the file processing device by the user through a terminal.

[0093] It should be noted that the first lexical file can be obtained first, followed by the second lexical file; or the second lexical file can be obtained first, followed by the first lexical file; or the first and second lexical files can be obtained simultaneously. This application embodiment does not limit the execution order of obtaining the first and second lexical files.

[0094] In one feasible implementation, the first lexical file can be a lexical file for the C language, and the second lexical file can be a lexical file for C++.

[0095] It should be noted that only the first grammar file and the second grammar file can be obtained; only the first lexical file and the second lexical file can be obtained; or both the first grammar file and the second grammar file, as well as the first lexical file and the second lexical file, can be obtained. Step 101 can be executed after step 102, or before step 102, or steps 101 and 102 can be executed simultaneously. In this embodiment, the execution order of steps 101 and 102 is not limited.

[0096] Step 103: If the first language and the second language are determined to be compatible, determine the target lexical file based on the first lexical file and the second lexical file.

[0097] The target lexical file includes a file that integrates the lexical structure of the first language and the lexical structure of the second language.

[0098] In this embodiment of the application, the file processing device can analyze the grammatical structure of the first grammar file and the second language file, and determine whether the first language and the second language have language compatibility conditions based on the analysis results. If the analysis results indicate that the first language and the second language have compatibility conditions, then the first language and the second language are made compatible. The compatibility of the first language and the second language is mainly reflected in the compatibility of the grammar of the first language and the grammar of the second language, as well as the compatibility of the lexical structure of the first language and the lexical structure of the second language.

[0099] In this embodiment, a first lexical file and a second lexical file can be analyzed, and the first lexical file and the second lexical file can be merged based on the analysis results to obtain a target lexical file. This target lexical file contains both the lexical data of the first language and the lexical data of the second language, so that subsequent lexical analysis of programs (code) written in the first language and programs written in the second language can be performed using only the target lexical file, thereby improving the efficiency of lexical analysis of programs.

[0100] Lexical analysis, in computer science, is the process of converting a sequence of characters into a sequence of tokens. The program or function that performs lexical analysis is called a lexical analyzer or scanner, and it is generated based on the target lexical file. The lexical analyzer typically exists as a function, which is called by the syntax analyzer.

[0101] Step 104: If the first language and the second language are determined to be compatible, determine the target grammar file based on the first grammar file and the second grammar file.

[0102] The target grammar file includes a file that integrates the grammar of the first language and the grammar of the second language.

[0103] In this embodiment of the application, the grammatical structure of the first language in the first grammar file and the grammatical structure of the second language in the second grammar file can be analyzed to obtain the analysis results. Based on the analysis results, the first grammar file and the second grammar file are merged to generate a target grammar file, so that the target grammar file contains the grammar of the first language and the grammar of the second language. This allows the grammar of the program written in the first language and the grammar of the program written in the second language to be analyzed using only the target grammar file, thereby improving the efficiency of grammar analysis.

[0104] In one feasible implementation, the target grammar file is a file that integrates the syntax of both C and C++ languages. Therefore, subsequent programs written in either C or C++ can be analyzed using this target grammar file. Syntax analysis, based on lexical analysis, combines word sequences into various grammatical phrases such as "program," "statement," and "expression." Syntax analysis uses a parser to determine the structural correctness of the program. The program's structure is described by a context-free grammar, and the parser is generated based on the target grammar file.

[0105] Step 105: Process the file to be processed based on the target lexical file and / or target syntax file.

[0106] The file to be processed is a file containing code. This code can be in an unknown language.

[0107] In this embodiment of the application, a target lexical file can be used to analyze the words of the unknown language code in the file to be processed, and a target syntax file can be used to analyze the statements of the unknown language code in the file to be processed, so as to determine the meaning of the unknown language code, determine whether the unknown language code is malicious code based on the meaning of the unknown language code, and then determine whether the file to be processed containing the malicious code is a malicious file.

[0108] The file processing method provided in this application embodiment obtains a first grammar file for a first language and a second grammar file for a second language; and / or obtains a first lexical file for the first language and a second lexical file for the second language; when it is determined that the first language and the second language have language compatibility, a target lexical file is determined based on the first lexical file and the second lexical file; wherein the target lexical file includes a file that integrates the lexicals of the first language and the second language; and / or, when it is determined that the first language and the second language have language compatibility, a target grammar file is determined based on the first grammar file and the second grammar file; wherein the target grammar file includes a file that integrates the grammars of the first language and the grammars of the second language; and the file to be processed is processed based on the target lexical file and / or the target grammar file; thus, by using a target grammar file that integrates the grammars of multiple languages ​​and / or a target lexical file that integrates the lexicals of multiple languages ​​to process the file to be processed, it is not necessary to try to process the file to be processed using the grammar file and lexical file of each language one by one, which improves the efficiency of processing the file to be processed, and also eliminates the need to load multiple grammar files and multiple lexical files when processing the file to be processed, further reducing development costs.

[0109] Based on the foregoing embodiments, this application provides a file processing method, referring to... Figure 2 The method includes the following steps:

[0110] Step 201: The file processing device acquires a first grammar file for the first language and a second grammar file for the second language.

[0111] It should be noted that the first syntax file can be obtained first, followed by the second syntax file; or the second syntax file can be obtained first, followed by the first syntax file; or the first and second syntax files can be obtained simultaneously. In this embodiment, the execution order of obtaining the first and second syntax files is not limited.

[0112] Step 202: The file processing device determines the complexity of the first language based on the first grammar structure in the first grammar file, and determines the complexity of the second language based on the second grammar structure in the second grammar file.

[0113] The first grammatical structure refers to the four major structures of the grammar (also known as syntax) of a first language; the second grammatical structure refers to the four major structures of the grammar of a second language. The four major structures include sequence structure, loop structure, branch structure, and function structure. Branch structure can also be called selection structure. The four major structures can also be called the grammatical structure of a language.

[0114] In this embodiment of the application, the file processing device can parse a first grammar file to obtain the first sequential structure, first loop structure, first branch structure, and first function structure of the four major structures in the first grammar file, and evaluate the complexity of the first language based on the number of first sequential structures, the number of first loop structures, the number of first branch structures, and the number of first function structures; the file processing device can parse a second grammar file to obtain the second sequential structure, second loop structure, second branch structure, and second function structure of the four major structures in the second grammar file, and evaluate the complexity of the second language based on the number of second sequential structures, the number of second loop structures, the number of second branch structures, and the number of second function structures.

[0115] Specifically, the complexity of a first language can be assessed based on the numerical differences between the four major structures in the first grammar file; and the complexity of a second language can be assessed based on the numerical differences between the four major structures in the second grammar file.

[0116] In one feasible implementation, taking the first language as an example, the number of first sequential structures in the first grammar file is 'a', the number of first loop structures is 'b', the number of first branch structures is 'c', and the number of first function structures is 'd'. The variances of a, b, c, and d can be calculated, and the calculated variances can be used as the complexity of the first language. The complexity of the second language can be obtained using the same method. In another feasible implementation, the weights of the first sequential structures, first loop structures, first branch structures, and first function structures can be set separately, and the weight of each structure can be multiplied by its respective quantity. Then, the values ​​obtained by multiplying the weights of multiple structures by their respective quantities are summed to obtain the complexity of the first language.

[0117] It should be noted that, based on the number of the four major structural elements in the first language and the second language, one language can be chosen as the primary language. The grammar and lexical structure of the primary language serve as the framework, and the grammar and lexical structure of the non-primary language can be integrated into it. Since the primary language needs good extensibility and maintainability, it is not advisable to favor any one of the four major structural elements. Instead, a language where the number of the four structural elements is relatively similar can be selected as the primary language. There can be multiple first languages, multiple second languages, and only one primary language.

[0118] Step 203: Determine the language similarity between the first grammar file and the second grammar file based on the similarity of the identifiers in the first grammar file and the identifiers in the second grammar file.

[0119] Identifiers may include, but are not limited to, statement identifiers, function identifiers, and code block identifiers; statement identifiers, function identifiers, and code block identifiers are sets of identifiers of different categories determined after classifying basic identifiers.

[0120] In this embodiment, a first weight parameter for statement identifiers, a second weight parameter for function identifiers, and a third weight parameter for code block identifiers can be set. The first similarity between the first statement identifier in the first grammar file and the second statement identifier in the second grammar file, the second similarity between the first function identifier in the first grammar file and the second function identifier in the second grammar file, and the third similarity between the first code block identifier in the first grammar file and the second code block identifier in the second grammar file are calculated. The first similarity is multiplied by the first weight parameter to obtain a first value, the second similarity is multiplied by the second weight parameter to obtain a second value, and the third similarity is multiplied by the third weight parameter to obtain a third value. Then, the first value, the second value, and the third value are summed to obtain the language similarity between the first grammar file and the second grammar file.

[0121] It should be noted that the first, second, and third values ​​can also be averaged to obtain an average value, which can then be used as the language similarity between the first and second syntax files. Identifiers can also include language character sets and symbol interpretations.

[0122] If the first language is C and the second language is Java, since Java is based on C, Java's syntax is basically the same as C's in terms of the four major structures. Both use ";" as the statement terminator, ";" as the closing semicolon, "{}" as the code block identifier, parentheses as the function identifier, and "," as the separator between parameters. The complexity of C and Java is also basically the same. Therefore, if we only consider language compatibility, Java and C will be classified into the same language family.

[0123] In one feasible implementation, the language similarity between the first and second syntax files can be achieved by converting them into syntax trees and analyzing them. Each leaf node of the syntax tree represents a basic sentence structure. A symbol can be abstracted from these basic structures, and a mapping address is assigned to each symbol in a vector space to represent the degree of similarity between these basic structures (e.g., assignment statements and arithmetic operations are both mathematical expressions, function definitions and class definitions are both code blocks; code blocks and mathematical expressions would become two different clusters in the vector space). Having obtained the symbol represented by each leaf node, a sequence of symbols can be obtained for each upper-level syntax structure in the syntax tree. Since the similarity between each symbol has been obtained through distance in the vector space, the similarity calculation of the symbol sequences of the top-level syntax structures can yield the degree of similarity between any two languages. This method can provide a preliminary assessment of some newly added or unknown languages, analyzing similar features to facilitate subsequent analysis.

[0124] Step 204: The file processing device determines whether the first language and the second language have language compatibility conditions based on the complexity of the first language, the complexity of the second language, and the language similarity.

[0125] It should be noted that determining whether a first language and a second language are compatible requires considering both the complexity of each language and their differences. These differences lie in the four main structural aspects of the first and second languages, and their similarity in these structural aspects can be used to determine their compatibility.

[0126] In this embodiment, the compatibility of a first language with other languages ​​can be determined based on the complexity of the first language, and the compatibility of a second language with other languages ​​can be determined based on the complexity of the second language. If both the first and second languages ​​are compatible, their compatibility can be determined based on their language similarity. A higher similarity between the first and second languages ​​indicates a smaller difference between them, and thus, greater compatibility.

[0127] Step 204 can be achieved through the following steps:

[0128] a. If the complexity of the first language and the complexity of the second language are both no greater than the complexity threshold, and the language similarity is greater than the similarity threshold, the file processing device determines that the first language and the second language have language compatibility conditions.

[0129] The complexity threshold is a pre-set critical threshold used to measure whether the grammatical structure of a language is complex.

[0130] In the embodiments of this application, when the complexity of the first language and the complexity of the second language are both not greater than the complexity threshold, it indicates that the first language and the second language have the conditions for initial compatibility with other languages. When the language similarity between the first language and the second language is greater than the similarity threshold, that is, when the similarity between the first language and the second language is high, it indicates that the first language and the second language have the conditions for language compatibility, so that the first language and the second language can be compatible in the future.

[0131] b. If the complexity of the first language or the second language is not greater than the complexity threshold, or the language similarity is not greater than the language similarity threshold, the file processing device determines that the first language and the second language do not meet the language compatibility conditions.

[0132] It should be noted that when the complexity of any language exceeds the complexity threshold, it indicates that the language is highly complex and not suitable for compatibility with other languages. Conversely, when the complexity of any language is not greater than the complexity threshold, it indicates that the language is not highly complex and is suitable for compatibility with other languages.

[0133] In the embodiments of this application, if the complexity of the first language or the complexity of the second language is greater than the complexity threshold, it indicates that the first language or the second language does not meet the language compatibility conditions; or if the complexity of the first language and the complexity of the second language are not greater than the complexity threshold, and the language similarity between the first language and the second language is not greater than the language similarity threshold, then although the complexity of the first language and the second language is not high, the differences between the first language and the second language are large and they are also suitable for compatibility.

[0134] Step 205: If the first language and the second language are not compatible, the file processing device generates and outputs alarm information.

[0135] The alarm message is used to indicate that the first language and the second language are incompatible.

[0136] In this embodiment, when it is determined that the first language and the second language do not meet the language compatibility requirements, it indicates that compatibility processing between the first language and the second language cannot be performed. An alarm message can then be generated to notify the user that the first language and the second language are incompatible, allowing the user to promptly determine the reason for the incompatibility based on the alarm message. The alarm message may further include the complexity of the first language, the complexity of the second language, and the similarity between the first language and the second language.

[0137] It should be noted that the alarm information can also be used to prompt users to add syntax and lexical files for other languages ​​to ensure compatibility with those languages.

[0138] Step 206: The file processing device acquires the first lexical file and the second lexical file when the first language and the second language are compatible.

[0139] It should be noted that if the first and second languages ​​are compatible, then obtaining the first lexical file of the first language and the second lexical file of the second language can avoid the situation where the first and second languages ​​are incompatible, which would cause the obtained first and second lexical files to occupy a lot of memory and increase the workload of the file processing device. This further reduces the workload of the file processing device and reduces the memory occupied space.

[0140] The specific process of obtaining the first lexical file and the second lexical file is similar to step 102, and will not be repeated here in this embodiment.

[0141] It should be noted that the first lexical file can be obtained first, followed by the second lexical file; or the second lexical file can be obtained first, followed by the first lexical file; or the first and second lexical files can be obtained simultaneously. This application embodiment does not limit the execution order of obtaining the first and second lexical files.

[0142] Step 207: The file processing device analyzes the first lexical file to determine the first word, and analyzes the second lexical file to determine the second word.

[0143] In this context, both the first and second words can be represented by tokens. A token can also be understood as a word obtained by segmenting the program's statements.

[0144] In this embodiment of the application, the program (or code) in the first lexical file may be analyzed, and the program (or code) in the first lexical file may be segmented to obtain the first word. The program in the second lexical file may be analyzed, and the program in the second lexical file may be segmented to obtain the second word.

[0145] Step 208: The document processing device determines the first word information of the first word and the second word information of the second word.

[0146] The first word information includes, but is not limited to, the first regular expression of the first word; the second word information includes, but is not limited to, the second regular expression of the second word.

[0147] It should be noted that the file processing device can retrieve the first word information corresponding to the first word from the first lexical file based on the first word, and retrieve the second word information corresponding to the second word from the second lexical file based on the second word. The file processing device can also query the lexical database for the first word information corresponding to the first word, and query the lexical database for the second word information corresponding to the second word. Specifically, the first lexical file contains the first word and its corresponding first word information, and the second lexical file contains the second word and its corresponding second word information.

[0148] Step 208 can be implemented in the following way:

[0149] a1. The file processing device determines the first regular expression of the first word from the first lexical file and the second regular expression of the second word from the second lexical file.

[0150] In this embodiment of the application, a regular expression corresponding to each first word can be determined from multiple regular expressions in a first lexical file based on the first word, and a second regular expression for the second word can be determined from multiple regular expressions in a second lexical file based on the second word.

[0151] Step 209: The file processing device determines the target lexical file based on the first word, the second word, the first word information, and the second word information.

[0152] In this embodiment of the application, it can be determined whether the first language and the second language are lexically compatible based on the first word information and the second word information. If the first language and the second language are lexically compatible, the first word and the second word are fused together, and the first word information and the second word information are fused together to obtain the target lexical file.

[0153] Step 209 can be achieved through the following steps:

[0154] a2. The file processing device determines the target lexical file based on the first word, the second word, the first regular expression, and the second regular expression.

[0155] In this embodiment, a first regular expression and a second regular expression can be matched. Based on the matching degree, the types of the first and second words, it is determined whether the lexical structures of the first and second languages ​​are compatible. If the lexical structures of the first and second languages ​​are determined to be compatible, the first and second words are merged, and the first and second regular expressions are also merged to obtain a target lexical file. The target lexical file contains both the lexical structures of the first and second languages.

[0156] Wherein, a2 can be achieved through steps b1-b5:

[0157] b1. The file processing device determines the matching degree of the first regular expression and the second regular expression.

[0158] In this embodiment of the application, a first regular expression and a second regular expression can be matched, and the degree of matching between the first regular expression and the second regular expression can be determined.

[0159] b2. When the matching degree is the target threshold, the file processing device sets the first lexical type of the first word and the second lexical type of the second word to the same lexical type, and determines the target lexical file based on the first word, the second word and the first regular expression.

[0160] The target threshold can be 100%.

[0161] In this embodiment of the application, when the matching degree is 100%, that is, when the first regular expression and the second regular expression are completely identical, the first lexical type of the first word and the second lexical type of the second word can be set to the same lexical type, and the first word and the second word can be merged, and the first regular expression and the second regular expression can be merged to obtain the target lexical file; wherein, when the first regular expression and the second regular expression are completely identical, the regular expression in the target lexical file obtained by merging the first regular expression and the second regular expression can also be understood as the first regular expression or the second regular expression.

[0162] b3. The file processing device determines the target lexical file based on the first word, the second word, the first regular expression, and the second regular expression when the matching degree is greater than zero and less than the target threshold, and the first lexical type and the second lexical type are the same, or when the matching degree is zero.

[0163] In this embodiment, when the matching degree is greater than zero and less than 100%, and the first lexical type and the second lexical type are the same (i.e., the regular expressions of similar lexical tokens have overlap), the first word and the second word can be merged, and the first regular expression and the second regular expression can be merged to obtain the target lexical file. When the matching degree is zero (i.e., the first regular expression and the second regular expression are completely different), the first word and the second word can be merged, and the first regular expression and the second regular expression can be merged to obtain the target lexical file.

[0164] b4. When the matching degree is greater than zero and less than the target threshold, and the first lexical type and the second lexical type are different, the file processing device obtains the first association relationship between the first words and the second association relationship between the second words, and determines the target lexical file based on the first words, the second words, the first association relationship, the second association relationship, the first regular expression and the second regular expression.

[0165] In this embodiment of the application, when the matching degree is greater than zero and less than the target threshold, and the first lexical type and the second lexical type are different, that is, when the regular expressions of different types of tokens have an intersection, it is necessary to determine the first meaning of the first word based on the first word and the first association relationship, and determine the second meaning of the second word based on the second word and the second association relationship. Based on the first meaning and the second meaning, the first word and the second word are merged, and the first regular expression and the second regular expression are merged to obtain the target lexical file.

[0166] It should be noted that when regular expressions of different types of tokens overlap, word segmentation conflicts often occur. By determining the first meaning of the first word based on the first word and the first association, and determining the second meaning of the second word based on the second word and the second association (essentially, judging the semantic meaning of the word based on its context), this approach is often quite efficient, and word segmentation conflicts often vary greatly and are easily identified.

[0167] b5. When the matching degree is greater than zero and less than the target threshold, and the first lexical type and the second lexical type are different, the file processing device determines the first lexical path based on the first word and the first regular expression, and determines the second lexical path based on the second word and the second regular expression.

[0168] The target lexical file includes a first lexical path and a second lexical path.

[0169] If the matching degree is greater than zero and less than the target threshold, and the first and second lexical types are different (meaning there is an overlap in the regular expressions of different types of tokens), the first lexical path can be determined based on the first word and the first regular expression, and the second regular expression can be used to determine the first regular expression path. When performing lexical analysis on the file to be processed using the target lexical file, the first lexical path in the target lexical file can be used for lexical analysis. If the analysis using the first lexical path fails, the second lexical path can be used for lexical analysis. In other words, different lexical paths are used to attempt parsing the file to be processed. If a lexical error occurs, another semantic approach is tried until success is achieved. Each analysis failure eliminates one language from the unknown code in the file to be detected.

[0170] Step 210: The file processing device obtains the first statement in the first syntax file and the second statement in the second syntax file.

[0171] In this embodiment of the application, a first statement can be determined from a first syntax file and a second statement can be determined from a second syntax file based on a statement identifier.

[0172] Step 211: The file processing device determines the target syntax file based on the first and second statements.

[0173] In this embodiment of the application, the syntax of the first statement can be determined by analyzing the first statement and the syntax of the second statement can be determined by analyzing the second statement. The syntax of the first statement and the syntax of the second statement can be merged to determine the target syntax file.

[0174] Step 211 can also be achieved through steps c1-c3:

[0175] c1. The file processing device obtains the first grammatical formula of the first statement and the second grammatical formula of the second statement.

[0176] In this embodiment of the application, the first grammatical constraint of each first statement can be determined from multiple grammatical constraints in multiple first grammar files, and the second grammatical constraint of each second statement can be determined from multiple grammatical constraints in a second grammar file. Each first statement in the first grammar file corresponds to a first grammatical constraint; each second statement in the second grammar file corresponds to a second grammatical constraint.

[0177] It should be noted that the first linguistic legal formula can be obtained first, followed by the second linguistic legal formula; or the second linguistic legal formula can be obtained first, followed by the first linguistic legal formula; or the first and second linguistic legal formulas can be obtained simultaneously. The embodiments of this application do not limit the execution order of obtaining the first and second linguistic legal formulas.

[0178] c2. The document processing equipment determines the first sentence structure association between the sentence structures corresponding to the first grammatical contract and the second sentence structure association between the sentence structures corresponding to the second grammatical contract;

[0179] Among them, the first linguistic convention can be obtained based on multiple first sentence patterns; the second linguistic convention can be obtained based on multiple second sentence patterns.

[0180] In the embodiments of this application, multiple first sentence patterns in the first grammatical constraint can be analyzed to determine the relationship between the first sentence patterns, and multiple second sentence patterns in the second grammatical constraint can be analyzed to determine the relationship between the second sentence patterns.

[0181] In one feasible implementation, a first syntax tree can be generated based on a first syntax file, and a second syntax tree can be generated based on a second syntax file. The first grammar rules and first sentence structure relationships are determined based on the leaf nodes of the first syntax tree, and the second grammar rules and second sentence structure relationships are determined based on the leaf nodes of the second syntax tree. Each leaf node in the syntax tree represents a basic sentence structure.

[0182] c3. The file processing device combines the first and second sentences based on the first sentence association relationship and the second sentence association relationship, and combines the second grammar constraint with the first grammar constraint to obtain the target grammar file.

[0183] In this embodiment of the application, the first statement and the second statement can be combined based on the first sentence structure association relationship and the second association relationship, and the second grammatical constraint and the first grammatical constraint can be combined based on the first sentence structure association relationship and the second association relationship to obtain the target grammar file.

[0184] In one feasible implementation, the level of each first grammar rule can be determined in the first syntax tree corresponding to the first grammar file, and the level of each second grammar rule can be determined from the first syntax tree corresponding to the second grammar file. The second grammar rule with the same level as the first grammar rule can be added to the level where the first grammar rule is located. That is, the second grammar rule can be added as a branch of the first grammar rule in the first syntax tree.

[0185] c3 can be implemented by d1-d3:

[0186] d1. The file processing device combines the first and second sentences based on the first sentence association relationship and the second sentence association relationship, and combines the second grammar rule with the first grammar rule to obtain alternative grammar files.

[0187] The implementation process of step d1 is similar to that of step c3, and will not be described again in this embodiment.

[0188] d2. If there are grammatical errors in the candidate grammar file, the file processing device corrects the candidate grammar file to obtain the target grammar file.

[0189] In this embodiment of the application, the candidate syntax file can be compiled to generate a state transition table, and the state transition table can be used to determine whether there are syntax errors in the candidate syntax file. When there are syntax errors, the erroneous syntax parts in the candidate syntax file are corrected, and the corrected candidate syntax file is used as the target syntax file.

[0190] Wherein, d2 can be achieved through steps e1, e2, e3, and e4:

[0191] e1. The file processing device adjusts the grammatical constraints in the candidate grammar files to obtain the target grammar file.

[0192] In this embodiment of the application, adjusting the grammatical constraints in the candidate grammar file may refer to adjusting the position of the grammatical constraints in the candidate grammar file to obtain the target grammar file.

[0193] In one feasible implementation, the file processing device can generate a candidate syntax tree based on the candidate syntax file, and adjust the level of the grammatical rules in the candidate syntax tree so that the level of the adjusted grammatical rules is higher than that of the unadjusted grammatical rules, thereby improving the generalization ability of the grammar to be compatible with the grammar of the language.

[0194] e2. If there are errors in the candidate syntax file, the file processing device deletes the erroneous content in the candidate syntax file and obtains the target syntax file.

[0195] In this embodiment of the application, an error recovery mechanism can be used to skip erroneous content in the candidate file, or it can be understood as deleting erroneous content in the candidate syntax file to obtain the target syntax file.

[0196] e3. If there are errors in the candidate grammar file, the file processing device increases the length of the third grammar rule in the candidate grammar file to obtain the target grammar file.

[0197] In this embodiment of the application, if there is an error in the candidate grammar file, the length of the third grammar rule in the candidate grammar file can be increased to make the length of the third grammar rule in the candidate grammar file as long as possible, thereby reducing the reduction path length. In this way, it can be avoided that the grammar rule cannot locate the reduction path in the current grammar state when looking at a token, which would cause the processing of the file to fail.

[0198] e4. When there are errors in the candidate grammar files, the file processing device obtains the grammatical relationships between the third-level grammar rules in the candidate grammar files, and determines the target grammar file based on the grammatical relationships and the third-level grammar rules.

[0199] In this embodiment, when an error exists in a candidate grammar file, the file processing device obtains the grammatical relationships between third-degree grammar rules in the candidate grammar files and determines the target grammar file based on these relationships and the third-degree grammar rules. That is, the target grammar file is determined by judging the grammar that should be used in the context of the candidate grammar files.

[0200] It should be noted that when making the grammars of the first language compatible with those of the second language, for ambiguous grammars, high-frequency grammars can be prioritized for compatibility based on their usage frequency, while very low-frequency grammars can be discarded. For the main language, priority should be given to retaining them, while other non-main languages ​​can be appropriately discarded.

[0201] d3. If there are no syntax errors in the candidate syntax file, the file processing device determines the target syntax file as the candidate syntax file.

[0202] In this embodiment of the application, if there are no grammatical errors in the candidate grammar file, then the candidate grammar file can be directly used as the target grammar file.

[0203] In this embodiment of the application, compatibility between the first language and the second language can also be achieved through a model:

[0204] In one feasible implementation, such as Figure 3 As shown, a first grammar file and a second grammar file can be input into a language compatibility evaluation model. The model outputs the language similarity between the first and second languages, the complexity of the first language, and the complexity of the second language. It determines whether the complexity of both languages ​​is less than a complexity threshold, and if the language similarity is greater than the similarity threshold, then the first and second lexical files are input into a lexical compatibility evaluation model to perform lexical compatibility on the first and second languages. Similarly, the first and second grammar files are input into a grammar compatibility evaluation model to perform grammatical compatibility on the first and second languages. When lexical conflicts exist during lexical compatibility on the first and second languages, a lexical conflict resolution model can be used to resolve these conflicts, ultimately generating the target lexical file. When grammatical conflicts exist during grammatical compatibility on the first and second languages, a grammatical conflict resolution model can be used to resolve these conflicts, ultimately generating the target grammar file. It should be noted that the process of generating the target lexical file can be considered as a process of simultaneously compatibility and conflict resolution between the lexical of the first language and the lexical of the second language; the process of generating the target syntax file can be considered as a process of simultaneously compatibility and conflict resolution between the syntax of the first language and the syntax of the second language.

[0205] Step 212: The file processing device acquires the file to be processed, which carries the code.

[0206] The file to be processed can be a file generated by code intercepted by the firewall.

[0207] In this embodiment, a firewall installed on the file processing device may intercept code that intrudes into the file processing device and generate a file to be processed based on the code; alternatively, a firewall installed on the server may intercept code that intrudes into the server and generate a file to be processed based on the code, and then send the file to be processed to the file processing device.

[0208] Step 213: The file processing device detects words in the code based on the target lexical file and determines the first detection result.

[0209] In this embodiment of the application, the file processing device can analyze the words in the code based on the target lexical file, identify the part of speech and meaning of the words in the code, and obtain the first detection result.

[0210] Step 214: The file processing device detects the statements in the code based on the target syntax file and determines the second detection result.

[0211] In this embodiment of the application, the file processing device can analyze the statements in the code based on the target syntax file, identify the semantics of the statements in the code, and obtain a second detection result.

[0212] Step 215: The file processing device determines whether the file to be processed is a malicious file based on the first detection result and / or the second detection result.

[0213] In this embodiment of the application, the file processing device can determine the meaning of the code based on the part of speech and meaning of the words in the code, and / or the semantics of the statements in the code, and then analyze whether the code is malicious code based on the meaning of the code. If the code is determined to be malicious code, the file to be processed containing the malicious code is determined to be a malicious file.

[0214] It should be noted that the descriptions of the same steps and contents as in other embodiments in this embodiment can be found in the descriptions in other embodiments, and will not be repeated here.

[0215] The file processing method provided in this application uses a target grammar file that integrates the grammars of multiple languages ​​and / or a target lexical file that integrates the lexicals of multiple languages ​​to process the file to be processed. This eliminates the need to try processing the file to be processed using the grammar file and lexical file of each language one by one, thereby improving the efficiency of processing the file to be processed. Furthermore, it eliminates the need to load multiple grammar files and multiple lexical files when processing the file to be processed, further reducing development costs.

[0216] Based on the foregoing embodiments, this application provides a file processing method, referring to... Figure 4 The method includes the following steps:

[0217] Step 301: The file processing device acquires syntax files for at least two of the multiple languages.

[0218] Among them, any two languages ​​are two languages ​​randomly selected from a variety of languages.

[0219] In this application embodiment, the multiple languages ​​may include, but are not limited to, C, C++, Java, and Python. In one feasible implementation, any two languages ​​may be C and C++.

[0220] It should be noted that the process of obtaining grammar files for at least two of the multiple languages ​​in step 301 is similar to the process in step 101, and will not be described again in this embodiment.

[0221] Step 302: The file processing device acquires lexical files for at least two of the multiple languages.

[0222] In the embodiments of this application, at least two languages ​​are the same as the at least two languages ​​in "obtaining grammar files for at least two languages ​​among multiple languages"; in one feasible implementation, if the obtained grammar files are for C language and C++, then the obtained lexical files are also lexical files for C language and C++.

[0223] It should be noted that the process of obtaining lexical files for at least two of the multiple languages ​​in step 302 is similar to the process in step 102, and will not be described again in this embodiment. Step 301 can be executed before step 302, or after step 302, or steps 301 and 302 can be executed simultaneously. In this embodiment, the execution order of steps 301 and 302 is not limited.

[0224] Step 303: The file processing device determines the target lexical file based on the lexical files of at least two languages, provided that at least two languages ​​meet the language compatibility requirements.

[0225] The target lexical file includes a file that incorporates lexical data from at least two languages.

[0226] Step 304: The file processing device determines the target syntax file based on the syntax files of at least two languages, provided that at least two languages ​​are compatible.

[0227] The target grammar file includes a file that incorporates the grammars of at least two languages.

[0228] Step 305: The file processing device processes the file to be processed based on the target lexical file and / or target syntax file.

[0229] It should be noted that the implementation process of steps 303-305 is the same as that of steps 103-105, and will not be described again in this embodiment.

[0230] The file processing method provided in this application embodiment obtains grammar files for at least two languages ​​from a variety of languages; and / or obtains lexical files for at least two languages ​​from a variety of languages; if it is determined that at least two languages ​​have language compatibility conditions, a target lexical file is determined based on the lexical files of at least two languages; wherein the target lexical file includes a file that integrates the lexical files of at least two languages; and / or, if it is determined that at least two languages ​​have language compatibility conditions, a target grammar file is determined based on the grammar files of at least two languages; wherein the target grammar file includes a file that integrates the grammar files of at least two languages; and the file to be processed is processed based on the target lexical file and / or the target grammar file; thus, by using a target grammar file that integrates the grammar files of at least two languages ​​from a variety of languages ​​and / or a target lexical file that integrates the lexical files of multiple languages ​​to process the file to be processed, it is not necessary to try to process the file to be processed using the grammar files and lexical files of each language one by one, which improves the efficiency of processing the file to be processed, and also eliminates the need to load multiple grammar files and multiple lexical files when processing the file to be processed, further reducing development costs.

[0231] Based on the foregoing embodiments, this application provides a file processing method, referring to... Figure 5 The method includes the following steps:

[0232] Step 401: The file processing device acquires a first grammar file for the first language and a second grammar file for the second language.

[0233] It should be noted that the implementation process of step 401 is the same as that of step 101, and will not be described again in this embodiment.

[0234] Step 402: The file processing device determines the complexity of the first language based on the first grammar structure in the first grammar file, and determines the complexity of the second language based on the second grammar structure in the second grammar file.

[0235] Step 403: The file processing device determines the language similarity between the first grammar file and the second grammar file based on the similarity between the identifiers in the first grammar file and the identifiers in the second grammar file.

[0236] Step 404: The file processing device determines whether the first language and the second language have language compatibility conditions based on the complexity of the first language, the complexity of the second language, and the language similarity.

[0237] It should be noted that the implementation process of steps 402-404 is the same as that of steps 202-204, and will not be described again in this embodiment.

[0238] The file processing method provided in this application embodiment obtains a first grammar file for a first language and a second grammar file for a second language; determines the complexity of the first language based on the first grammar structure in the first grammar file, and determines the complexity of the second language based on the second grammar structure in the second grammar file; determines the language similarity between the first grammar file and the second grammar file based on the similarity between identifiers in the first grammar file and identifiers in the second grammar file; and determines whether the first language and the second language have language compatibility conditions based on the complexity of the first language, the complexity of the second language, and the language similarity. Thus, determining whether the first language and the second language have language compatibility conditions based on the complexity of the first language, the complexity of the second language, and the language similarity improves the accuracy of determining whether the first language and the second language have language compatibility conditions.

[0239] Based on the foregoing embodiments, this application also provides a first document processing apparatus, which can be applied to... Figure 1-2 In the corresponding embodiment of the file processing method, refer to Figure 6 The first document processing device 5 includes:

[0240] The first acquisition unit 51 is configured to acquire a first grammar file for a first language and a second grammar file for a second language; and / or,

[0241] The first acquisition unit 51 is also used to acquire a first lexical file for a first language and a second lexical file for a second language;

[0242] The first determining unit 52 is configured to determine a target lexical file based on the first lexical file and the second lexical file, provided that the first language and the second language are determined to be compatible; wherein the target lexical file includes a file that integrates the lexical structures of the first language and the second language; and / or,

[0243] The first determining unit 52 is further configured to determine a target grammar file based on the first grammar file and the second grammar file, provided that the first language and the second language are determined to be compatible; wherein the target grammar file includes a file that integrates the grammar of the first language and the grammar of the second language.

[0244] The first processing unit 53 is used to process the file to be processed based on the target lexical file and / or the target syntax file.

[0245] In this embodiment of the application, the first determining unit 52 is further configured to perform the following steps:

[0246] Get the first word from the first lexical file and the second word from the second lexical file;

[0247] Determine the first word information of the first word and the second word information of the second word;

[0248] The target lexical file is determined based on the first word, the second word, the information of the first word, and the information of the second word.

[0249] In this embodiment of the application, the first determining unit 52 is further configured to perform the following steps:

[0250] Determine the first regular expression for the first word from the first lexical file, and determine the second regular expression for the second word from the second lexical file;

[0251] The target lexical file is determined based on the first word, the second word, the first regular expression, and the second regular expression.

[0252] In this embodiment of the application, the first determining unit 52 is further configured to perform the following steps:

[0253] Determine the matching degree between the first regular expression and the second regular expression;

[0254] When the matching degree is the target threshold, the first lexical type of the first word and the second lexical type of the second word are set to the same lexical type, and the target lexical file is determined based on the first word, the second word and the first regular expression;

[0255] If the matching degree is greater than zero and less than the target threshold, and the first lexical type and the second lexical type are the same, or if the matching degree is zero, the target lexical file is determined based on the first word, the second word, the first regular expression, and the second regular expression.

[0256] In this embodiment of the application, the first determining unit 52 is further configured to perform the following steps:

[0257] In cases where the matching degree is greater than zero and less than the target threshold, and the first lexical type and the second lexical type are different, the first association relationship between the first words and the second association relationship between the second words are obtained, and the target lexical file is determined based on the first words, the second words, the first association relationship, the second association relationship, the first regular expression and the second regular expression.

[0258] In this embodiment of the application, the first determining unit 52 is further configured to perform the following steps:

[0259] In cases where the matching degree is greater than zero and less than the target threshold, and the first lexical type and the second lexical type are different, the first lexical path is determined based on the first word and the first regular expression, and the second lexical path is determined based on the second word and the second regular expression; wherein, the target lexical file includes the first lexical path and the second lexical path.

[0260] In this embodiment of the application, the first determining unit 52 is further configured to perform the following steps:

[0261] Get the first statement in the first syntax file and the second statement in the second syntax file;

[0262] Based on the first and second statements, the target syntax file is determined.

[0263] In this embodiment of the application, the first determining unit 52 is further configured to perform the following steps:

[0264] Obtain the first grammatical constraint of the first statement and the second grammatical constraint of the second statement;

[0265] Determine the first sentence structure relationship between the sentence structures corresponding to the first linguistic normative form, and determine the second sentence structure relationship between the sentence structures corresponding to the second linguistic normative form;

[0266] Based on the first and second sentence structure relationships, the first and second sentences are combined, and the second grammatical constraint and the first grammatical constraint are combined to obtain the target grammar file.

[0267] In this embodiment of the application, the first determining unit 52 is further configured to perform the following steps:

[0268] Based on the first sentence structure association relationship and the second sentence structure association relationship, the first sentence and the second sentence are combined, and the second grammar rule and the first grammar rule are combined to obtain the alternative grammar file;

[0269] If there are grammatical errors in the candidate grammar file, the candidate grammar file is corrected to obtain the target grammar file;

[0270] If there are no syntax errors in the candidate syntax file, the target syntax file is selected as the candidate syntax file.

[0271] In this embodiment of the application, the first determining unit 52 is further configured to perform the following steps:

[0272] Adjust the third-order grammar rules in the candidate grammar files to obtain the target grammar file; or

[0273] If the candidate syntax file contains errors, delete the erroneous content in the candidate syntax file to obtain the target syntax file;

[0274] If there are errors in the alternative grammar file, increase the length of the third grammar rule in the alternative grammar file to obtain the target grammar file;

[0275] If there are errors in the candidate grammar files, obtain the grammatical relationships between third-language legal norms, and determine the target grammar file based on the grammatical relationships and third-language legal norms.

[0276] In this embodiment of the application, the first acquisition unit 51 is further configured to perform the following steps:

[0277] If the first and second languages ​​are compatible, obtain the first lexical file and the second lexical file;

[0278] If the first language and the second language are not compatible, an alarm message is generated and output. The alarm message is used to indicate that the first language and the second language are incompatible.

[0279] In this embodiment of the application, the first determining unit 52 is further configured to perform the following steps:

[0280] The complexity of the first language is determined based on the first grammar structure in the first grammar file, and the complexity of the second language is determined based on the second grammar structure in the second grammar file.

[0281] The language similarity between the first grammar file and the second grammar file is determined based on the similarity of the identifiers in the first grammar file and the identifiers in the second grammar file.

[0282] Based on the complexity of the first language, the complexity of the second language, and the language similarity, determine whether the first and second languages ​​meet the conditions for language compatibility.

[0283] In this embodiment of the application, the first determining unit 52 is further configured to perform the following steps:

[0284] If the complexity of the first language and the complexity of the second language are both no greater than the complexity threshold, and the language similarity is greater than the similarity threshold, then the first language and the second language are determined to be compatible.

[0285] If the complexity of the first language or the complexity of the second language exceeds a complexity threshold, or the language similarity does not exceed a language similarity threshold, then the first language and the second language are determined to be not compatible.

[0286] It should be noted that the interaction process between the units in the embodiments of this application can be referred to Figure 1-2 The implementation process of the file processing method provided in the corresponding embodiments will not be described in detail here.

[0287] The first file processing apparatus provided in this application uses a target grammar file that integrates the grammars of multiple languages ​​and a target lexical file that integrates the lexicals of multiple languages ​​to process the file to be processed. It does not require trying to process the file to be processed with the grammar file and lexical file of each language one by one, which improves the efficiency of processing the file to be processed. Moreover, it also eliminates the need to load multiple grammar files and multiple lexical files when processing the file to be processed, further reducing development costs.

[0288] Based on the foregoing embodiments, this application also provides a second document processing apparatus, which can be applied to... Figure 4 In the corresponding embodiment of the file processing method, refer to Figure 7 The second document processing device 6 includes:

[0289] The second acquisition unit 61 acquires grammar files for at least two of the multiple languages; and / or acquires lexical files for at least two of the multiple languages.

[0290] The second determining unit 62, upon determining that the at least two languages ​​meet the language compatibility requirements, determines a target lexical file based on the lexical files of the at least two languages; wherein the target lexical file includes a file that integrates the lexical files of the at least two languages; and / or,

[0291] The second determining unit 62 is further configured to determine a target grammar file based on the grammar files of the at least two languages, provided that the at least two languages ​​are determined to have language compatibility; wherein the target grammar file includes a file that integrates the grammars of the at least two languages.

[0292] The second determining unit 63 is further configured to process the file to be processed based on the target lexical file and / or the target syntax file.

[0293] It should be noted that the interaction process between the units in the embodiments of this application can be referred to Figure 4 The implementation process of the file processing method provided in the corresponding embodiments will not be described in detail here.

[0294] The second file processing apparatus provided in this application uses a target grammar file that integrates the grammars of at least two languages ​​from multiple languages ​​and / or a target lexical file that integrates the lexicals of multiple languages ​​to process the file to be processed. This eliminates the need to try processing the file to be processed using the grammar file and lexical file of each language one by one, thereby improving the efficiency of processing the file to be processed. Furthermore, it eliminates the need to load multiple grammar files and multiple lexical files when processing the file to be processed, further reducing development costs.

[0295] Based on the foregoing embodiments, this application also provides a third document processing apparatus, which can be applied to... Figure 5 In the corresponding embodiment of the file processing method, refer to Figure 8 The third document processing device 7 includes:

[0296] The third acquisition unit 71 is used to acquire a first grammar file for the first language and a second grammar file for the second language;

[0297] The third determining unit 72 is used to determine the complexity of the first language based on the first grammar structure in the first grammar file, and to determine the complexity of the second language based on the second grammar structure in the second grammar file.

[0298] The third determining unit 72 is further configured to determine the language similarity between the first grammar file and the second grammar file based on the similarity between the identifiers in the first grammar file and the identifiers in the second grammar file.

[0299] The third determining unit 72 is further configured to determine whether the first language and the second language have language compatibility conditions based on the complexity of the first language, the complexity of the second language, and the language similarity.

[0300] It should be noted that the interaction process between the units in the embodiments of this application can be referred to Figure 5 The implementation process of the file processing method provided in the corresponding embodiments will not be described in detail here.

[0301] The third document processing apparatus provided in this application determines whether the first language and the second language have language compatibility conditions based on the complexity of the first language, the complexity of the second language, and language similarity, thereby improving the accuracy of determining whether the first language and the second language have language compatibility conditions.

[0302] Based on the foregoing embodiments, embodiments of this application provide a file processing device that can be applied to... Figure 1-2 and Figure 4-5 In the corresponding embodiment of the file processing method, refer to Figure 9 As shown, the memory processing device 8 may include: a processor 81, a memory 82, and a communication bus 83, wherein:

[0303] Communication bus 83 is used to realize the communication connection between processor 81 and memory 82;

[0304] Processor 81 is used to execute the file processing program in memory 82 to achieve Figure 1-2 and Figure 4-5The steps in the file processing method provided in the corresponding embodiment.

[0305] It should be noted that the specific implementation process of the steps executed by the processor in this embodiment can be referred to... Figure 1-2 and Figure 4-5 The implementation process of the file processing method provided in the corresponding embodiment will not be described in detail here.

[0306] The file processing device provided in the embodiments of this application uses a target grammar file that integrates the grammars of multiple languages ​​and / or a target lexical file that integrates the lexicals of multiple languages ​​to process the file to be processed. It does not require trying to process the file to be processed with the grammar file and lexical file of each language one by one, which improves the efficiency of processing the file to be processed. Moreover, it eliminates the need to load multiple grammar files and multiple lexical files when processing the file to be processed, and further reduces the development cost.

[0307] Based on the foregoing embodiments, embodiments of this application provide a computer-readable storage medium storing one or more programs, which can be executed by one or more processors to implement... Figure 1-2 and Figure 4-5 The steps in the file processing method provided in the corresponding embodiment.

[0308] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0309] It should be noted that the aforementioned computer-readable storage medium may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), or an electrically erasable programmable read-only memory.

[0310] Memory such as Electrically Erasable Programmable Read-Only Memory (EEPROM), Ferromagnetic Random Access Memory (FRAM), Flash Memory, Magnetic Surface Memory, Optical Disc, or Compact Disc Read-Only Memory (CD-ROM); or various electronic devices including one or any combination of the above memory, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0311] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0312] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0313] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0314] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0315] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0316] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0317] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A file processing method, characterized in that, The method includes: Obtain a first syntax file for a first language and a second syntax file for a second language; wherein the first language and the second language are programming languages; Obtain a first lexical file for the first language and a second lexical file for the second language; If it is determined that the first language and the second language are compatible, a target lexical file is determined based on the first lexical file and the second lexical file; wherein, the target lexical file includes a file that integrates the lexicals of the first language and the lexicals of the second language; If it is determined that the first language and the second language are compatible, a target grammar file is determined based on the first grammar file and the second grammar file; wherein, the target grammar file includes a file that integrates the grammar of the first language and the grammar of the second language; The file to be processed is processed based on the target lexical file and / or the target syntax file.

2. The method according to claim 1, characterized in that, The step of determining the target lexical file based on the first lexical file and the second lexical file includes: Obtain the first word from the first lexical file and the second word from the second lexical file; Determine the first word information of the first word and the second word information of the second word; The target lexical file is determined based on the first word, the second word, the information of the first word, and the information of the second word.

3. The method according to claim 1, characterized in that, The step of determining the target syntax file based on the first syntax file and the second syntax file includes: Obtain the first statement from the first grammar file and the second statement from the second grammar file; The target syntax file is determined based on the first statement and the second statement.

4. The method according to claim 3, characterized in that, The step of determining the target syntax file based on the first statement and the second statement includes: Obtain the first grammatical formula of the first statement and the second grammatical formula of the second statement; Determine the first sentence structure relationship between the sentence structures corresponding to the first grammatical constraint, and determine the second sentence structure relationship between the sentence structures corresponding to the second grammatical constraint; Based on the first sentence structure association relationship and the second sentence structure association relationship, the first statement and the second statement are combined, and the second grammatical constraint and the first grammatical constraint are combined to obtain the target grammar file.

5. The method according to claim 1, characterized in that, The step of obtaining the first lexical file for the first language and the second lexical file for the second language includes: If the first language and the second language are compatible, obtain the first lexical file and the second lexical file; Accordingly, the method further includes: If the first language and the second language are not compatible, an alarm message is generated and output, wherein the alarm message is used to indicate that the first language and the second language are incompatible.

6. The method according to claim 1 or 5, further comprising, before determining the target lexical file based on the first lexical file and the second lexical file after determining that the first language and the second language have language compatibility conditions: The complexity of the first language is determined based on the first grammar structure in the first grammar file, and the complexity of the second language is determined based on the second grammar structure in the second grammar file. Based on the similarity between the identifiers in the first grammar file and the identifiers in the second grammar file, the language similarity between the first grammar file and the second grammar file is determined. Based on the complexity of the first language, the complexity of the second language, and the language similarity, it is determined whether the first language and the second language meet the language compatibility criteria.

7. The method according to claim 6, characterized in that, The step of determining whether the first language and the second language meet the language compatibility criteria based on the complexity of the first language, the complexity of the second language, and the language similarity includes: If the complexity of the first language and the complexity of the second language are both no greater than the complexity threshold, and the language similarity is greater than the similarity threshold, then the first language and the second language are determined to have language compatibility. If the complexity of the first language or the complexity of the second language is greater than the complexity threshold, or the language similarity is not greater than the language similarity threshold, it is determined that the first language and the second language do not meet the language compatibility conditions.

8. The method according to claim 1, characterized in that, The processing of the file to be processed based on the target lexical file and / or the target syntax file includes: Retrieve the file to be processed containing code; Based on the target lexical file, words in the code are detected to determine a first detection result; and / or, Based on the target syntax file, the statements in the code are detected to determine the second detection result; Based on the first detection result and / or the second detection result, determine whether the file to be processed is a malicious file.

9. A file processing method, characterized in that, The method includes: Obtain syntax files for at least two of a plurality of languages; wherein the plurality of languages ​​are programming languages; Obtain lexical files for at least two of the multiple languages; If it is determined that at least two languages ​​meet the language compatibility requirements, a target lexical file is determined based on the lexical files of the at least two languages; wherein, the target lexical file includes a file that integrates the lexical files of the at least two languages; If it is determined that at least two languages ​​meet the language compatibility requirements, a target grammar file is determined based on the grammar files of the at least two languages; wherein, the target grammar file includes a file that integrates the grammars of the at least two languages; The file to be processed is processed based on the target lexical file and / or the target syntax file.

10. A file processing method, characterized in that, The method includes: Obtain a first syntax file for a first language and a second syntax file for a second language; wherein the first language and the second language are programming languages; The complexity of the first language is determined based on the first grammar structure in the first grammar file, and the complexity of the second language is determined based on the second grammar structure in the second grammar file. Based on the similarity between the identifiers in the first grammar file and the identifiers in the second grammar file, the language similarity between the first grammar file and the second grammar file is determined. Based on the complexity of the first language, the complexity of the second language, and the language similarity, determine whether the first language and the second language have language compatibility conditions; The method further includes: If it is determined that the first language and the second language are compatible, a target grammar file is determined based on the first grammar file and the second grammar file, and the file to be processed is processed based on the target grammar file; the target grammar file includes a file that integrates the grammar of the first language and the grammar of the second language.

11. A document processing device, characterized in that, The device includes: a processor, a memory, and a communication bus; The communication bus is used to realize the communication connection between the processor and the memory; The processor is used to execute a file processing program in memory to implement the steps of the file processing method as described in any one of claims 1 to 10.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, which can be executed by one or more processors to implement the steps of the file processing method as described in any one of claims 1 to 10.

Images