Software detection method, device and equipment and computer storage medium

Abstract

The invention provides a software detection method, device and equipment and a computer storage medium, and the method relates to the technical field of computers, and is used for improving the efficiency and objectivity of assessment score evaluation in practical operation assessment. The method comprises the following steps: dividing a program file of software to be detected into at least two data blocks; respectively acquiring the distribution condition of at least one continuous byte sequence set in each data block in one data block; the byte lengths corresponding to the continuous byte sequence sets are different; based on the obtained distribution condition corresponding to each continuous byte sequence set, performing classification identification processing on one data block, and obtaining a classification identification result corresponding to one data block; and based on the obtained classification and identification results corresponding to the at least two data blocks, combining the position information of each data block in the program file, and carrying out weighted fusion on the obtained classification and identification results to obtain a target detection result corresponding to the to-be-detected software.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, apparatus and device for software detection, and a computer storage medium. Background Art

[0002] With the rapid development of information technology, computer systems and networks have become an integral part of daily life and industrial operations. However, this has also posed security challenges, with the threat of malware being a particularly prominent and growing threat. Malware, which encompasses viruses, worms, Trojan horses, and other malicious code, can disrupt system functionality, steal sensitive data, and even completely control equipment.

[0003] Malware detection methods primarily rely on signature matching technology, where security software identifies threats by comparing signatures with known malware. Specifically, software signatures typically use a hash function to calculate a hash value for the software or data. This hash value is then encrypted using the developer's private key to generate a digital signature. This digital signature is then attached to the software or data package. When a user receives the software or data, they decrypt the digital signature using the corresponding public key, obtaining a hash value. This hash value is then compared with the current hash value of the software or data. If the two match, the software or data is considered trustworthy.

[0004] However, this method has significant limitations: First, it relies on prior knowledge. Signature matching technology requires malware samples before it can generate corresponding feature signatures. This method cannot effectively detect newly emerging or mutated malware, especially zero-day malware. Second, it is susceptible to obfuscation technology. Attackers can easily bypass signature-based detection systems with the help of various obfuscation technologies (such as code encryption and deformation). Third, it has a high false alarm rate. In some cases, legitimate software may have similar characteristics to malware, resulting in false alarms and increasing system maintenance costs.

[0005] Therefore, there is an urgent need for a new software detection method to improve the efficiency and accuracy of malicious software detection. Summary of the Invention

[0006] Embodiments of the present application provide a software detection method, apparatus, device, and computer storage medium to improve the efficiency and accuracy of malicious software detection.

[0007] In a first aspect, an embodiment of the present application provides a method for software detection, comprising:

[0008] Obtaining software to be tested, and dividing the program file of the software to be tested into at least two data blocks; wherein each data block has the same byte size;

[0009] For each of the at least two data blocks, perform the following operations:

[0010] Obtaining a distribution of at least one continuous byte sequence set in a data block, wherein the continuous byte sequence is any continuous byte of a fixed byte length in the data block, and each continuous byte sequence set corresponds to a different byte length;

[0011] Based on the obtained distribution of each set of continuous byte sequences, a classification and recognition process is performed on the data block to obtain a classification and recognition result corresponding to the data block; wherein the classification and recognition result represents: a probability that the data block belongs to a preset target category;

[0012] Based on the classification and recognition results corresponding to the at least two data blocks obtained, combined with the position information of each data block in the program file, the at least two classification and recognition results obtained are weightedly fused to obtain the target detection result corresponding to the software to be detected.

[0013] Optionally, obtaining a distribution of at least one continuous byte sequence set in a data block in the data block includes:

[0014] Extracting at least one continuous byte sequence set from all bytes contained in the one data block; wherein the byte length corresponding to each continuous byte sequence set is one of the preset byte length sets;

[0015] For the at least one continuous byte sequence set, respectively obtaining the occurrence frequency of each continuous byte sequence in each continuous byte sequence set in the corresponding continuous byte sequence set;

[0016] Based on the obtained occurrence frequency corresponding to each continuous sequence of bytes, the distribution of each continuous byte sequence set in the data block is extracted.

[0017] Optionally, for the at least one continuous byte sequence set, obtaining the occurrence frequency of each continuous byte sequence in each continuous byte sequence set in the corresponding continuous byte sequence set includes:

[0018] For each of the at least one continuous byte sequence set, perform the following operations respectively:

[0019] Obtaining the number of occurrences of each continuous byte sequence in a continuous byte sequence set;

[0020] Based on the number of occurrences and the total number of continuous byte sequences in the continuous byte sequence set, the occurrence frequency of each continuous byte sequence in the continuous byte sequence set is obtained.

[0021] Optionally, the distribution situation includes at least one or more of the following:

[0022] A Simpsons index determined based on the frequency of occurrence of each of the consecutive sequence bytes;

[0023] The Canberra distance is determined based on the frequency of occurrence of two consecutive sequence bytes;

[0024] Minkowski distance determined based on the frequency of occurrence of two adjacent consecutive sequence bytes;

[0025] Manhattan distance determined based on the frequency of occurrence of two consecutive sequence bytes;

[0026] The Chebyshev distance is determined based on the frequency of occurrence of two adjacent consecutive sequence bytes;

[0027] Bray-Curtis distance based on the frequency of occurrence of two consecutive sequence bytes;

[0028] Angular separation determined based on the frequency of occurrence of two adjacent consecutive sequence bytes;

[0029] A correlation coefficient determined based on the occurrence frequency corresponding to each continuous sequence of bytes and the average occurrence probability of the corresponding continuous sequence of bytes;

[0030] The total variation determined based on the frequency of occurrence of two consecutive sequence bytes;

[0031] An information entropy determined based on the frequency of occurrence of each consecutive sequence of bytes;

[0032] Kuhler-Leibler divergence based on the frequency of occurrence of two consecutive sequence bytes;

[0033] Jensen-Shannon divergence based on the frequency of occurrence of two consecutive sequence bytes;

[0034] The Ito-Saito divergence is determined based on the frequency of occurrence of two adjacent consecutive sequence bytes.

[0035] Optionally, performing classification and identification processing on the data block based on the obtained distribution corresponding to each set of continuous byte sequences to obtain a classification and identification result corresponding to the data block includes:

[0036] Performing classification and recognition processing on the data block based on the obtained distribution of each continuous byte sequence set using the trained classification and recognition model to obtain a classification and recognition result corresponding to the data block;

[0037] The classification recognition model includes at least two sub-classification identifiers, and each sub-classification identifier is used to perform classification recognition processing on a preset classification in the preset classification set.

[0038] Optionally, performing classification and recognition processing on the data block based on the obtained distribution corresponding to each continuous byte sequence set using a trained classification and recognition model to obtain a classification and recognition result corresponding to the data block includes:

[0039] Combining the obtained distribution conditions corresponding to each set of continuous byte sequences into the data distribution characteristics corresponding to the one data block;

[0040] Performing classification and recognition processing on the data distribution feature by each of the at least two sub-classification identifiers, and obtaining a sub-recognition result corresponding to each sub-classification identifier;

[0041] Based on the weight coefficient preset for each sub-classification identifier, the at least two sub-recognition results obtained are weightedly fused to obtain a classification recognition result corresponding to the one data block.

[0042] Optionally, the step of performing weighted fusion on the at least two classification recognition results obtained based on the classification recognition results corresponding to the at least two data blocks obtained, combined with the position information of each data block in the program file, to obtain the target detection result corresponding to the software to be detected includes:

[0043] Based on the position information of each data block in the program file, obtaining a position weight corresponding to each data block; the position weight is used to represent the importance of the corresponding data block in the program file;

[0044] Based on the acquired position weight corresponding to each data block, weighted fusion processing is performed on the classification recognition results corresponding to each data block to obtain the target detection result corresponding to the software to be detected.

[0045] Optionally, performing weighted fusion processing on the classification recognition results corresponding to each data block based on the acquired position weight corresponding to each data block to obtain the target detection result corresponding to the software to be detected includes:

[0046] Based on the acquired position weights corresponding to each data block, weighted fusion processing is performed on the classification recognition results corresponding to each data block to obtain a corresponding intermediate detection result;

[0047] Based on the classification and recognition results, obtaining, in all data blocks, a number of data blocks continuously characterized by the classification and recognition results as belonging to the preset target category;

[0048] Based on the quantity value, the intermediate detection result is corrected to obtain the target detection result corresponding to the software to be detected.

[0049] In a second aspect, an embodiment of the present application provides a software detection device, comprising:

[0050] An acquisition module is used to acquire the software to be detected and divide the program file of the software to be detected into at least two data blocks; wherein each data block has the same byte size;

[0051] an identification module configured to perform the following operations for each of the at least two data blocks: obtaining a distribution of at least one continuous byte sequence set in the data block, wherein the continuous byte sequence is any continuous byte of a fixed byte length in the data block, and each continuous byte sequence set has a different byte length; performing classification and identification processing on the data block based on the obtained distribution of each continuous byte sequence set, and obtaining a classification and identification result corresponding to the data block; wherein the classification and identification result represents a probability that the data block belongs to a preset target category;

[0052] The processing module is used to perform weighted fusion on the at least two classification recognition results obtained based on the classification recognition results corresponding to the at least two data blocks obtained, combined with the position information of each data block in the program file, to obtain the target detection result corresponding to the software to be detected.

[0053] Optionally, the identification module is configured to obtain at least one set of continuous byte sequences in a data block, and when determining the distribution of the set in the data block, is specifically configured to:

[0054] Extracting at least one continuous byte sequence set from all bytes contained in the one data block; wherein the byte length corresponding to each continuous byte sequence set is one of the preset byte length sets;

[0055] For the at least one continuous byte sequence set, respectively obtaining the occurrence frequency of each continuous byte sequence in each continuous byte sequence set in the corresponding continuous byte sequence set;

[0056] Based on the obtained occurrence frequency corresponding to each continuous sequence of bytes, the distribution of each continuous byte sequence set in the data block is extracted.

[0057] Optionally, the identification module is configured to obtain, for the at least one continuous byte sequence set, respectively the occurrence frequency of each continuous byte sequence in each continuous byte sequence set, specifically for:

[0058] For each of the at least one continuous byte sequence set, perform the following operations respectively:

[0059] Obtaining the number of occurrences of each continuous byte sequence in a continuous byte sequence set;

[0060] Based on the number of occurrences and the total number of continuous byte sequences in the continuous byte sequence set, the occurrence frequency of each continuous byte sequence in the continuous byte sequence set is obtained.

[0061] Optionally, the distribution situation includes at least one or more of the following:

[0062] A Simpsons index determined based on the frequency of occurrence of each of the consecutive sequence bytes;

[0063] The Canberra distance is determined based on the frequency of occurrence of two consecutive sequence bytes;

[0064] Minkowski distance determined based on the frequency of occurrence of two adjacent consecutive sequence bytes;

[0065] Manhattan distance determined based on the frequency of occurrence of two consecutive sequence bytes;

[0066] The Chebyshev distance is determined based on the frequency of occurrence of two adjacent consecutive sequence bytes;

[0067] Bray-Curtis distance based on the frequency of occurrence of two consecutive sequence bytes;

[0068] Angular separation determined based on the frequency of occurrence of two adjacent consecutive sequence bytes;

[0069] A correlation coefficient determined based on the occurrence frequency corresponding to each continuous sequence of bytes and the average occurrence probability of the corresponding continuous sequence of bytes;

[0070] The total variation determined based on the frequency of occurrence of two consecutive sequence bytes;

[0071] An information entropy determined based on the frequency of occurrence of each consecutive sequence of bytes;

[0072] Kuhler-Leibler divergence based on the frequency of occurrence of two consecutive sequence bytes;

[0073] Jensen-Shannon divergence based on the frequency of occurrence of two consecutive sequence bytes;

[0074] The Ito-Saito divergence is determined based on the frequency of occurrence of two adjacent consecutive sequence bytes.

[0075] Optionally, the identification module is configured to perform classification and identification processing on the data block based on the obtained distribution corresponding to each continuous byte sequence set, and when obtaining the classification and identification result corresponding to the data block, specifically to:

[0076] Performing classification and recognition processing on the data block based on the obtained distribution of each continuous byte sequence set using the trained classification and recognition model to obtain a classification and recognition result corresponding to the data block;

[0077] The classification recognition model includes at least two sub-classification identifiers, and each sub-classification identifier is used to perform classification recognition processing on a preset classification in the preset classification set.

[0078] Optionally, the recognition module is configured to perform classification and recognition processing on the data block based on the distribution corresponding to each continuous byte sequence set obtained using a trained classification and recognition model, and to obtain a classification and recognition result corresponding to the data block, specifically for:

[0079] Combining the obtained distribution conditions corresponding to each set of continuous byte sequences into the data distribution characteristics corresponding to the one data block;

[0080] Performing classification and recognition processing on the data distribution feature by each of the at least two sub-classification identifiers, and obtaining a sub-recognition result corresponding to each sub-classification identifier;

[0081] Based on the weight coefficient preset for each sub-classification identifier, the at least two sub-recognition results obtained are weightedly fused to obtain a classification recognition result corresponding to the one data block.

[0082] Optionally, the processing module is configured to perform weighted fusion on the at least two classification and recognition results obtained based on the classification and recognition results corresponding to the at least two data blocks obtained, combined with the position information of each data block in the program file, to obtain the target detection result corresponding to the software to be detected, specifically for:

[0083] Based on the position information of each data block in the program file, obtaining a position weight corresponding to each data block; the position weight is used to represent the importance of the corresponding data block in the program file;

[0084] Based on the acquired position weight corresponding to each data block, weighted fusion processing is performed on the classification recognition results corresponding to each data block to obtain the target detection result corresponding to the software to be detected.

[0085] Optionally, the processing module is configured to perform weighted fusion processing on the classification recognition results corresponding to each data block based on the acquired position weight corresponding to each data block, and to obtain the target detection result corresponding to the software to be detected, specifically for:

[0086] Based on the acquired position weights corresponding to each data block, weighted fusion processing is performed on the classification recognition results corresponding to each data block to obtain a corresponding intermediate detection result;

[0087] Based on the classification and recognition results, obtaining, in all data blocks, a number of data blocks continuously characterized by the classification and recognition results as belonging to the preset target category;

[0088] Based on the quantity value, the intermediate detection result is corrected to obtain the target detection result corresponding to the software to be detected.

[0089] In a third aspect, an embodiment of the present application provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the at least one processor executes the instructions stored in the memory, thereby enabling the at least one processor to execute the method described in the first aspect or any optional embodiment of the first aspect.

[0090] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which is used to store instructions. When the instructions are executed, the method in the first aspect or any optional embodiment of the first aspect is implemented.

[0091] In a fifth aspect, an embodiment of the present application provides a computer program product comprising instructions, wherein the computer program product stores instructions that, when executed on a computer, enable the computer to execute the method described in the first aspect or any optional embodiment of the first aspect.

[0092] The beneficial effects of this application are as follows:

[0093] This solution performs targeted software type detection by segmenting the program files of the software being tested and performing byte-level analysis of the file contents. Because this solution is independent of file type, it is applicable to all file types and can be tested without prior knowledge of the file format, greatly enhancing its versatility and practicality.

[0094] In addition, this solution classifies and identifies data blocks based on the distribution of various sets of continuous byte sequences in the data blocks divided by program files. It can capture the inherent differences between the target software and normal files in the underlying byte-level content, thereby effectively detecting target software even when it appears for the first time or has been disguised.

[0095] On the other hand, this approach utilizes a trained classification and recognition model to classify and identify data blocks, not only improving the accuracy of software detection for pre-defined target categories but also enhancing the robustness of the system. Traditional methods are susceptible to various evasion techniques employed by malware developers, such as code obfuscation, encryption, or deformation. However, by utilizing a classification and recognition model, this approach continuously optimizes the sub-classifiers over multiple iterations, enabling them to better adapt to the diversity and variability of software within the pre-defined target categories. By adjusting the decision tree parameters and feature selection strategies, this approach can be further optimized to accommodate new types of software that may emerge in the future. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0097] Figure 1 A schematic diagram of possible application scenarios provided by the embodiments of the present application;

[0098] Figure 2 A flowchart of a software detection method provided in an embodiment of the present application;

[0099] Figure 3 A logical diagram of a method for dividing program files provided in an embodiment of the present application;

[0100] Figure 4 A logical diagram of a method for dividing a long sequence of text provided in an embodiment of the present application;

[0101] Figure 5 A flow chart of a method for obtaining distribution information provided in an embodiment of the present application;

[0102] Figure 6 A schematic diagram of a plurality of sets of consecutive byte sequences provided in an embodiment of the present application;

[0103] Figure 7 A training flow chart of the classification and recognition model provided in the embodiment of the present application;

[0104] Figure 8 A flowchart of a method for obtaining classification and recognition results provided in an embodiment of the present application;

[0105] Figure 9 A flowchart of a method for obtaining target detection results provided in an embodiment of the present application;

[0106] Figure 10 A schematic diagram of the structure of a target recognition device is provided for an embodiment of the present application;

[0107] Figure 11 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0108] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. Unless there is a conflict, the embodiments in the present application and the features in the embodiments can be combined with each other in any way. In addition, although a logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in an order different from that here.

[0109] The terms "first" and "second" in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any of its variations are intended to cover non-exclusive protection. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices. "Multiple" in this application can mean at least two, for example, two, three or more, and the embodiments of this application are not limited thereto.

[0110] The following description of exemplary embodiments of the present application is made in conjunction with the accompanying drawings, which include various details of the embodiments of the present application to facilitate understanding, and they should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope disclosed in this application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description. It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, which should be considered as exemplary, and their purpose is only to illustrate the feasibility of the implementation of the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0111] In the technical solution of this application, the acquisition, transmission, storage, and use of data comply with the requirements of relevant national laws and regulations.

[0112] With the rapid development of information technology, computer systems and networks have become an integral part of daily life and industrial operations. However, this has also posed security challenges, with the threat of malware being a particularly prominent and growing threat. Malware, which encompasses viruses, worms, Trojan horses, and other malicious code, can disrupt system functionality, steal sensitive data, and even completely control equipment.

[0113] Malware detection methods primarily rely on signature matching technology, where security software identifies threats by comparing signatures with known malware. Specifically, software signatures typically use a hash function to calculate a hash value for the software or data. This hash value is then encrypted using the developer's private key to generate a digital signature. This digital signature is then attached to the software or data package. When a user receives the software or data, they decrypt the digital signature using the corresponding public key, obtaining a hash value. This hash value is then compared with the current hash value of the software or data. If the two match, the software or data is considered trustworthy.

[0114] However, this method has significant limitations: First, it relies on prior knowledge. Signature matching technology requires malware samples before it can generate corresponding feature signatures. This method cannot effectively detect newly emerging or mutated malware, especially zero-day malware. Second, it is susceptible to obfuscation technology. Attackers can easily bypass signature-based detection systems with the help of various obfuscation technologies (such as code encryption and deformation). Third, it has a high false alarm rate. In some cases, legitimate software may have similar characteristics to malware, resulting in false alarms and increasing system maintenance costs.

[0115] In view of this, an embodiment of the present application provides a software detection method. After obtaining the software to be detected, the program file of the software to be detected can be directly divided into at least two data blocks, and the byte size corresponding to each data block is ensured to be the same. In this way, after obtaining at least two data blocks, the following operations can be performed on each data block:

[0116] The distribution of at least one continuous byte sequence set in a data block is determined within the data block, where the continuous byte sequence is any continuous byte of a fixed byte length in the data block, and each continuous byte sequence set corresponds to a different byte length. Thus, after obtaining different continuous byte sequence sets, the data block is classified and identified based on their corresponding distributions to obtain a classification identification result for the data block.

[0117] Finally, after performing the above operations on each data block, the classification and recognition results corresponding to each data block can be obtained. In this way, combined with the position information of each data block in the program file, all the classification and recognition results can be fused to obtain the target detection results corresponding to the software to be detected.

[0118] After completing the introduction of the main inventive ideas of the embodiments of the present application, the specific implementation of the software detection method proposed in the embodiments of the present application will be introduced below through some specific embodiments and drawings.

[0119] The following briefly introduces the application scenarios to which the technical solutions of the embodiments of the present application can be applied. It should be noted that the application scenarios described below are only used to illustrate the embodiments of the present application and are not limiting. In the specific implementation process, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0120] See also Figure 1 , which is a schematic diagram of a possible application scenario provided by an embodiment of the present application. In this scenario, a terminal device 101 and a server 102 may be included.

[0121] The terminal device 101 may be a mobile phone, tablet computer (PAD), personal computer (PC), wearable device, vehicle-mounted terminal, etc. The server 102 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It may also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0122] The terminal device 101 and the server 102 may be directly or indirectly connected to each other via one or more communication networks 103. The communication network 103 may be a wired network or a wireless network, for example, a mobile cellular network or a Wireless Fidelity (WIFI) network, or other possible networks, which are not limited in the embodiment of the present invention.

[0123] It should be noted that the software detection method in the embodiment of the present application can be Figure 1 The server 102 is used to execute the detection process. For example, the server may receive software to be detected from a communication-connected terminal device or other server, then divide the program file corresponding to the software to be detected into several data blocks of equal size. Based on the distribution of the continuous byte sequence sets within each of these data blocks, the server classifies and identifies each data block, obtaining a classification and identification result corresponding to each data block. Finally, the server can combine all the classification and identification results with the location information of each data block to obtain the target detection result for the corresponding software to be detected.

[0124] It should be noted that Figure 1 The examples shown are only for illustration. In fact, the number and communication mode of terminal devices and servers are not limited and are not specifically limited in the embodiments of this application.

[0125] On the other hand, the software detection method provided in the embodiments of the present application can be applied to a variety of software detection scenarios. For example, when a user downloads software that has not been reviewed in an application store, the software detection method can be used to detect the program files of the downloaded software to determine whether the software is safe software. Alternatively, for some zero-day malware, since it is newly emerged and has not yet been identified and prevented by security protection software, when the user downloads the software, the software detection method can be actively performed on this type of software to avoid the losses that may be caused by installing malware. It should be noted that the examples of the above application scenarios are all used to provide possible application scenarios of this solution, rather than specific limitations.

[0126] The following describes the software detection method provided by the exemplary embodiment of the present application in combination with the application scenarios described above and with reference to the accompanying drawings. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principles of the present application, and the implementation of the present application is not limited in this respect.

[0127] See also Figure 2 , is a flow chart of a software detection method provided by an embodiment of the present application. In order to facilitate the subsequent introduction of the method, the following will be Figure 1 The method is described below using the server shown as an example of an execution subject. Figure 2 As shown, the specific implementation steps of this method are as follows:

[0128] Step S201: obtaining software to be detected, and dividing the program file of the software to be detected into at least two data blocks; wherein the byte size of each data block is the same.

[0129] When a user wishes to test a piece of software, they can directly send the program files of the software to be tested to the server, which will then test the software. Alternatively, the user can send a download instruction to the server, which will automatically test the application during the download and installation process. In this way, whether the user actively initiates the test or the server automatically tests the downloaded application, the server can obtain the software to be tested for testing.

[0130] After obtaining the software to be detected, the server may divide the program file of the software to be detected into at least two data blocks, and the byte size of each data block is the same.

[0131] In this way, for the server, there is no need to determine the file type of the software to be tested, nor is there any need to know the file format in advance. Instead, the program files of the software to be tested are divided based solely on the byte size in the program files, which effectively improves the versatility and practicality of the solution.

[0132] For example, after receiving the software to be detected, the server may divide the program file F of the software to be detected into N data blocks of equal size, wherein each data block may be recorded as B i (i=1,2,…,N). Each data block B i Contains a portion of the byte sequence of file F, and the size of the data block is S bytes.

[0133] In one possible implementation, Figure 3 As shown, when dividing program files, the server can perform the following steps:

[0134] First, the server receives a program file F of the software to be detected, where the file size is |F| bytes.

[0135] The server can then divide the program file into chunks based on the byte size of each data chunk pre-set in the server. The byte size S pre-set in the server is a key parameter for controlling the granularity and performance of software detection. This parameter can be set to 1000 or can be specifically set based on user needs. This application does not impose any restrictions on this.

[0136] After the server determines that the byte size of each data block is S and receives the program file F, it can divide the program file F according to the byte size S of the data block to obtain N data blocks, where symbol Indicates that the result within the sign is rounded up.

[0137] Finally, the server can assign a unique identification ID to each data block obtained by division. i , for easy tracking and reference, and output each block B i For subsequent classification and identification processing.

[0138] In this way, after obtaining at least two data blocks, the server can perform the following operations on each of the at least two data blocks:

[0139] Step S202: Obtain at least one continuous byte sequence set in a data block and its distribution in the data block; wherein a continuous byte sequence is any continuous byte with a fixed byte length in a data block, and each continuous byte sequence set corresponds to a different byte length.

[0140] For each data block, the server can perform classification and identification processing on it separately. Specifically, it first extracts the distribution of data in a data block.

[0141] When extracting the distribution, the server targets at least one set of continuous byte sequences in the data block, so the following will first introduce the continuous byte sequences.

[0142] Continuous byte sequence (n-gram) refers to the process of breaking text into a sequence of consecutive n words (or bytes). For example, Figure 4 As shown in the figure, for a long text sequence, when obtaining continuous byte sequences from it, the corresponding continuous byte sequences can be obtained from the front to the back. For example, assuming a text sequence is: Jia, Yi, Bing, Ding, Wu, Ji, Geng, Xin, Ren, Gui, then when obtaining continuous byte sequences of length 1 from this sequence, the obtained sequences are: Jia, Yi, Bing, Ding, Wu, Ji, Geng, Xin, Ren, Gui; when obtaining continuous byte sequences of length 2 from this sequence, the obtained sequences are: Jia, Yi, Yi, Bing, Bing, Ding, Ding, Wu, Wu, Ji, Ji, Geng, Geng, Xin, Xin, Ren, Ren, Gui, these 9 sequences. Similarly, continuous byte sequences of length 3, length 4, length 5, etc. can also be obtained. For the convenience of recording, these continuous byte sequences of different lengths can be recorded as different types of continuous byte sequence sets and represented by 1-gram, 2-gram, 3-gram, 4-gram, etc.

[0143] After understanding the above content, it can be determined that when the server executes step S202, it can obtain at least one continuous byte sequence set from a data block, and further obtain the distribution of these continuous byte sequence sets in a data block.

[0144] In a possible implementation, when the server obtains the distribution status, it may complete the acquisition in the following manner.

[0145] See also Figure 5 , which is a flow chart of a method for obtaining distribution information provided in an embodiment of the present application, such as Figure 5 As shown, the specific implementation steps of this method are as follows:

[0146] Step S501: extracting at least one continuous byte sequence set from all bytes contained in a data block; wherein the byte length corresponding to each continuous byte sequence set is one of the preset byte length sets.

[0147] After obtaining a data block, the server needs to obtain the corresponding continuous byte sequence set from the data block. The method of obtaining the continuous byte sequence has been described in the above content and will not be repeated here in this application.

[0148] After obtaining the continuous byte sequence, the next step is to determine the content of at least one continuous byte sequence set.

[0149] First of all, it should be clear that a byte length set is pre-set in the server. The byte length contained in the byte length set is the length of the continuous byte sequence that the server needs to comply with when obtaining a continuous byte sequence. For example, the byte length set may include: 1 byte, 2 bytes, 3 bytes, 4 bytes, etc. Figure 6 As shown, when the server obtains a continuous byte sequence set, it needs to obtain a continuous byte sequence set corresponding to a 1-byte length, a continuous byte sequence set corresponding to a 2-byte length, a continuous byte sequence set corresponding to a 3-byte length, and a continuous byte sequence set corresponding to a 4-byte length.

[0150] For example, assuming that the server needs to obtain at least one continuous byte sequence set from a long sequence "012345678910" based on the above byte length set, it can obtain four continuous byte sequence sets, namely:

[0151] The set of consecutive byte sequences corresponding to 1 byte length is: {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 1, 0};

[0152] The set of consecutive byte sequences corresponding to 2-byte length is: {01, 12, 23, 34, 45, 56, 67, 78, 89, 91, 10};

[0153] The set of consecutive byte sequences corresponding to a 3-byte length is: {012, 123, 234, 345, 456, 567, 678, 789, 891, 910};

[0154] The set of consecutive byte sequences corresponding to a 4-byte length is: {0123, 1234, 2345, 3456, 4567, 5678, 6789, 7891, 8910}.

[0155] In this way, after the server extracts at least one set of continuous byte sequences for a data block, the server can continue to perform the following operations:

[0156] Step S502: for at least one continuous byte sequence set, respectively obtain the occurrence frequency of each continuous byte sequence in each continuous byte sequence set in the corresponding continuous byte sequence set.

[0157] After the server obtains the continuous byte sequence set, it needs to perform data processing on the continuous byte sequences in each continuous byte sequence set to obtain the occurrence frequency of each continuous byte sequence in the corresponding continuous byte sequence set.

[0158] In a possible implementation, the server may perform the following operations for each of the at least one continuous byte sequence set:

[0159] First, obtain the number of occurrences of each continuous byte sequence in a continuous byte sequence set;

[0160] Next, based on the occurrence index and the total number of continuous byte sequences included in a continuous byte sequence set, the occurrence frequency of each continuous byte sequence in a corresponding continuous direct sequence set is obtained.

[0161] For example, assume that for a specified data block B i , the server generates all consecutive byte sequences of length n (as mentioned above, n can be 1 or 2 or 3 or 4, or other lengths). If a data block B i The length of L is, then the jth continuous byte sequence (n-gram) g i,j It can be expressed as:

[0162] g i,j =SubByte(B i ,j,n)(Formula 1)

[0163] Here, j refers to a natural number from 1 to L-n+1.

[0164] Next, for each generated n-gram, count its occurrence count(g i,j ). Thus, the frequency F of each generated n-gram i,j The calculation formula is:

[0165]

[0166] Here, Formula 2 represents the number of times each generated n-gram appears divided by the total number of n-grams that may be generated in a data block, which is the corresponding occurrence frequency of the n-gram.

[0167] Optionally, in order to more intuitively represent the occurrence frequency of each generated continuous byte sequence, the server can also generate a corresponding frequency histogram for each set of continuous byte sequences, record the occurrence frequency of each continuous byte sequence, and use this histogram as input for subsequent feature calculations.

[0168] In this way, after obtaining the corresponding frequencies of all consecutive byte sequence sets, the server can continue to perform the following steps:

[0169] Step S503: Based on the obtained occurrence frequency corresponding to each consecutive byte, the distribution of each consecutive byte sequence set in a data block is extracted.

[0170] For each set of continuous byte sequences, the server can obtain its corresponding distribution. Specifically, within a set of continuous byte sequences, the server can obtain corresponding feature values ​​based on the obtained occurrence frequency of each continuous byte sequence in the set and, using the following feature extraction method, obtain one or more features of the distribution.

[0171] Specifically, the content included in the distribution situation may be one or more of the following. It should be noted that the content in the distribution situation can be divided into statistical features and information theory features according to the acquisition method. Among them, the statistical features include the following, namely:

[0172] Feature 1: Simpson's Index determined based on the frequency of occurrence of each consecutive sequence of bytes, namely:

[0173]

[0174] Among them, n j The frequency of occurrence of a continuous byte sequence in a set of continuous byte sequences, L is the total number of bytes in a data block. This index is used to measure the diversity of n-gram frequencies within a block.

[0175] Feature 2: Canberra distance determined based on the frequency of occurrence of two consecutive sequence bytes, namely:

[0176]

[0177] Among them, X j is the occurrence frequency of the jth continuous byte sequence in the continuous byte sequence set. This feature is used to measure the score of the coordinate difference between two objects.

[0178] Feature 3: Minkowski Distance of Orderλ, which is determined based on the frequency of occurrence of two consecutive sequence bytes, is:

[0179]

[0180] Among them, λ is usually set to 3, which is a generalized distance metric.

[0181] Feature 4: Manhattan distance determined based on the frequency of occurrence of two consecutive sequence bytes, namely:

[0182]

[0183] The Minkowski distance when λ=1 is the Manhattan distance.

[0184] Feature 5: Chebyshev distance determined based on the frequency of occurrence of two consecutive sequence bytes, namely:

[0185] ch j =max j |X j -X j+1 |(Formula 7)

[0186] This feature measures the distance of the maximum difference between the coordinates of two objects.

[0187] Feature 6: BrayCurtis distance determined based on the frequency of occurrence of two consecutive sequence bytes, namely:

[0188]

[0189] This feature is a normalized distance metric that measures the difference between two probability distributions.

[0190] Feature 7: Angular Separation:

[0191]

[0192] This feature measures the similarity between two vectors by taking the cosine of the angle between them.

[0193] Feature 8: Correlation coefficient determined based on the frequency of occurrence of each consecutive sequence of bytes and the average probability of occurrence of the corresponding consecutive sequence of bytes, that is:

[0194]

[0195] in, It is the average of the occurrence frequencies of each continuous byte sequence set in a data block. This feature is used to measure the similarity between two values.

[0196] Feature 9: Total Variation determined based on the frequency of occurrence of two consecutive sequence bytes, namely:

[0197]

[0198] The above are the statistical characteristics that may be included in the distribution. The following are the information theory characteristics that may be included in the distribution:

[0199] Feature 10: Entropy determined based on the frequency of occurrence of each consecutive sequence of bytes, namely:

[0200]

[0201] Here, R is a discrete random variable, and t(ri) is the frequency of occurrence of each continuous byte sequence in a set of continuous byte sequences in the data block. This feature is used to measure the degree of discreteness of a distribution.

[0202] Feature 10: Kullback-Leibler Divergence, determined based on the frequency of occurrence of two consecutive sequence bytes, namely:

[0203]

[0204] This feature is used to measure the difference between two probability distributions.

[0205] Feature 11: Jensen-Shannon Divergence, determined based on the frequency of occurrence of two consecutive sequence bytes, namely:

[0206]

[0207] in, This feature is used to measure the similarity between two probability distributions.

[0208] Feature 12: Itakura-Saito Divergence, determined based on the frequency of occurrence of two consecutive sequence bytes, namely:

[0209]

[0210] This feature represents a special Bregman distance that measures the maximum possible difference between two probability distributions.

[0211] In the above manner, the server can complete the collection of the distribution of each continuous byte sequence set in a data block. In a possible implementation, the byte length set may include four byte lengths: 1-byte length, 2-byte length, 3-byte length, and 4-byte length. In this way, the server will obtain 4 continuous byte sequence sets, and for these 4 continuous byte sequence sets, respectively obtain their corresponding distribution conditions. In this case, when the server obtains the above 13 features as information in the distribution for these 4 continuous byte sequence sets, the server can obtain a total of 52 types of feature information for describing the statistical and information theory characteristics of the data block, and thereby characterize the content characteristics of the data block, thereby facilitating classification and identification processing of the data block.

[0212] After obtaining the distribution of each set of continuous byte sequences in a data block, the server can continue to perform the following operations:

[0213] Step S203: Based on the obtained distribution of each continuous byte sequence set, a data block is classified and identified to obtain a classification and identification result corresponding to the data block; wherein the classification and identification result represents: the probability that the data block belongs to the preset target category.

[0214] After the server obtains the distribution of content features that can characterize the data blocks, it can perform classification and identification processing on the data blocks based on the distribution.

[0215] When the server performs classification and recognition processing, it can use the trained classification and recognition model to perform corresponding classification and recognition processing. The classification and recognition model can be a pre-trained classification and recognition model that is then deployed in the server for use by the server.

[0216] In a possible implementation, the classification recognition model may include at least two sub-classification identifiers, wherein each sub-classification identifier is used to perform classification recognition processing on a preset classification in a preset classification set.

[0217] For example, assuming the preset target category set in the server is malware, then corresponding sub-classification identifiers can be set in the classification recognition model for different types of malware. For example, possible malware types include backdoors, Trojans, viruses, scripts, or other types. Then, corresponding sub-classification identifiers can be set for each of these types, namely: sub-classification identifier 1 - backdoor, classification identifier 2 - Trojans, classification identifier 3 - viruses, classification identifier 4 - scripts, and classification identifier 5 - other. In this way, the distribution input into the classification recognition model will be input into different sub-classification identifiers for identification, thereby obtaining corresponding sub-identification results.

[0218] For the above classification recognition model including at least two sub-classification recognizers, the corresponding training process can be:

[0219] In one possible implementation, the embodiment of the present application introduces a boosted decision tree algorithm to improve the performance of each sub-classification identifier through multiple iterative learning. The server needs to perform the following operations:

[0220] See also Figure 7 , is a training flow chart of the classification recognition model provided in an embodiment of the present application, which includes the following steps:

[0221] Step S701: Acquire multiple training samples.

[0222] The multiple training samples proposed in this step may be positive samples and negative samples extracted in advance from program files of different software, wherein each training sample corresponds to its own true label.

[0223] Step S702: performing classification and recognition processing on at least one training sample of this iteration using the classification and recognition model to be trained in this iteration to obtain a corresponding sample recognition result.

[0224] In the embodiment of the present application, in each round of iterative training, all training samples can be used as the input of this round, or part of the training samples can be extracted as the input of this round. The classification and recognition process will be described in detail later, so it will not be described in detail here.

[0225] Step S703: Based on the sample recognition result of each training sample and the corresponding true label, determine the target loss value of the classification recognition model to be trained in this iteration.

[0226] Among them, the target loss value can be calculated using any possible loss function, such as the cross entropy loss function, whose formula is: Among them, y represents the true label, y ^ Characterize the prediction probability, or the mean squared error (MSE) loss function, the squared absolute error loss function, the maximum likelihood (LHL) loss function, etc. Of course, other possible loss functions can also be used, and the embodiment of the present application does not limit this.

[0227] Based on the difference between the sample recognition result and the true label corresponding to at least one training sample, and the loss weight corresponding to the type of operation action or device state represented by the true label, the target loss value corresponding to this round of iterative process is obtained.

[0228] Step S704: Determine whether the model to be trained in this iteration has reached the convergence condition.

[0229] In the embodiment of the present application, the convergence condition may include one or more of the following conditions:

[0230] (1) The number of iterations reaches the set threshold.

[0231] (2) The target loss value is less than the set loss threshold.

[0232] Step S705: If the determination process of step S704 is no, then the parameters of the classification recognition model to be trained are adjusted based on the target loss value.

[0233] In the embodiment of the present application, if the number of iterations does not exceed a preset number threshold and the model loss value is not less than the set loss threshold, then the determination process of step 704 can be determined to be negative, i.e., it is considered that the current model to be trained does not meet the convergence conditions, and it is necessary to adjust the model parameters to continue training. After the parameter adjustment, the next iterative training process is entered, i.e., jumping to step 702.

[0234] In a possible implementation, when the model still does not meet the convergence conditions, the model weight parameters can be updated by the gradient descent method, and the training can be continued with the updated model weight parameters.

[0235] When the number of iterations exceeds the preset threshold, or the model loss value is less than the set loss threshold, the determination process of step 704 is yes, that is, it is considered that the current recognition model to be trained has met the convergence conditions, the model training is completed, and the classification recognition model is obtained.

[0236] In a possible implementation, during any round of training, the server may further perform the following operations, namely:

[0237] A corresponding data block weight is set for each data block divided from the program file, and a corresponding identifier weight is set for each sub-classification identifier in the classification recognition model.

[0238] In this way, during multiple rounds of iterative training, the classification recognition model can be trained using training samples with data block weights, and each sub-classification identifier (denoted as h t ) corresponding to the recognition error rate (denoted as ε t ).

[0239] In this way, when adjusting the parameters of the classification recognition model, the identifier weight of the sub-classification identifier and the data block weight corresponding to the data block can also be adjusted based on the recognition error rate of each sub-classification identifier for data blocks at different positions in a program file. Among them, the data block weight corresponding to the data block that is incorrectly recognized by the sub-classification identifier can be appropriately increased.

[0240] In this way, the training efficiency of the classification recognition model can be further improved, and the recognition accuracy of the trained classification recognition model can be improved.

[0241] In order to verify the effectiveness of the trained classification and recognition model, the embodiment of the present application tests the classification and recognition model using the validation data in the validation set. The validation set can be at least one training sample from the above-mentioned multiple training samples that did not participate in the training process, or it can be other sample data with real labels in addition to the training samples. This application does not impose any restrictions on this.

[0242] If the evaluation index reaches the preset evaluation effect, the classification recognition model to be trained can be used in the actual classification recognition process. If the model evaluation fails, it is necessary to continue training and testing until the classification recognition model to be trained reaches the preset evaluation effect.

[0243] In a possible implementation, when the server performs corresponding classification and recognition processing using the classification and recognition model trained above, the server may complete the process in the following manner:

[0244] See also Figure 8, which is a flow chart of a method for obtaining classification recognition results provided in an embodiment of the present application, such as Figure 8 As shown, the specific implementation steps of this method are as follows:

[0245] Step S801: merging the obtained distribution conditions corresponding to each set of continuous byte sequences into a data distribution feature corresponding to a data block.

[0246] As mentioned above, for a data block, Figure 10 As shown, the server will obtain multiple sets of continuous byte sequences corresponding to the data block, and for each set of continuous byte sequences, it will obtain the corresponding distribution of multiple statistical feature information. Therefore, the server needs to merge these feature information to obtain the data distribution feature corresponding to the data block. For example, in the case where the server can obtain 52 types of feature information for a data block, when merging, the server can obtain a 52-dimensional feature P = [P1, P2, ..., P n ](n=52).

[0247] Step S802: performing classification and recognition processing on the data distribution features through each sub-classification identifier of at least two character classification identifiers in the classification and recognition model, and obtaining a sub-recognition result corresponding to each sub-classification identifier.

[0248] After obtaining the data distribution characteristics, the server can use each sub-classification identifier in the classification and recognition model to classify and recognize the data distribution characteristics. Since each sub-classification identifier is trained for classification and recognition of a specific type, the server can obtain the corresponding sub-recognition results from each sub-classification identifier.

[0249] For example, the server can obtain from sub-classification identifier 1-backdoor the sub-identification result of whether the data block is malicious data of the backdoor type, obtain from classification identifier 2-Trojan the sub-identification result of whether the data block is malicious data of the Trojan type, obtain from classification identifier 3-virus the sub-identification result of whether the data block is malicious data of the virus type, obtain from classification identifier 4-script the sub-identification result of whether the data block is malicious data of the script type, and obtain from classification identifier 5-other the sub-identification result of whether the data block is other types of malicious data.

[0250] Step S803: Based on the weight coefficient preset for each sub-classification identifier, weighted fusion is performed on the obtained at least two sub-classification identification results to obtain a classification identification result corresponding to a data block.

[0251] Among them, the preset weight coefficient can be directly set by the user based on his own experience or needs, or it can be the identifier weight obtained by the server according to the above training process for the sub-category identifier training. This application does not impose any restrictions on this.

[0252] After the server performs a weighted fusion on each of its sub-recognition results based on the weight coefficient, it can obtain the classification recognition result corresponding to the data block. The classification recognition result can be a probability value that the data block belongs to the preset target category, or a black-white discrimination result determined by the server based on the probability value and probability threshold, that is, 100% probability of belonging to the preset target category, or 0% probability of belonging to the preset target category.

[0253] In this approach, the introduction of an enhanced decision tree algorithm not only improves the accuracy of software detection for pre-defined target categories but also enhances the robustness of the system. Traditional methods are susceptible to various evasion techniques employed by malware developers, such as code obfuscation, encryption, or deformation. However, by using the enhanced decision tree algorithm, this approach continuously optimizes the sub-classifier identifier over multiple iterations, enabling it to better adapt to the diversity and variability of software within the pre-defined target categories. By adjusting the decision tree parameters and feature selection strategies, this solution can be further optimized to accommodate new types of software that may emerge in the future.

[0254] The above is the process of the server performing classification and identification processing on a data block. After the server performs the above classification and identification processing on all the data blocks of the at least two acquired data blocks, the server can continue to perform the following operations:

[0255] Step S204: Based on the classification and recognition results corresponding to the at least two acquired data blocks and combined with the position information of each data block in the program file, the at least two acquired classification and recognition results are weightedly fused to obtain the target detection result corresponding to the software to be detected.

[0256] After the server obtains the classification and identification results corresponding to all data blocks, it can perform weighted fusion processing on all the obtained classification and identification results according to the location information of each data block in the program file, so as to obtain the target detection result corresponding to the software to be detected, and use the target detection result to characterize whether the software to be detected is the target type software, where the target type software can be malware or sensitive software, etc., and this application does not impose any restrictions on this.

[0257] In a possible implementation, the server may obtain the target detection result in the following manner:

[0258] See also Figure 9, which is a flow chart of a method for obtaining target detection results provided in an embodiment of the present application, such as Figure 9 As shown, the specific implementation steps of this method are as follows:

[0259] Step S901: Based on the position information of each data block in the program file, obtain the position weight corresponding to each data block; wherein the position weight is used to represent the importance of the corresponding data block in the program file.

[0260] The server can set a corresponding position weight for each position in the program file based on the universal content corresponding to different file positions in the preset program file. For example, for the first few data blocks in the program file, they are more likely to contain important information, so the position weight corresponding to such positions will also be higher. Alternatively, the server can use the data block weights obtained in the above training process as the position weights corresponding to data blocks at different positions, and this application does not impose any restrictions on this.

[0261] Step S902: Based on the acquired position weights corresponding to each data block, weighted fusion processing is performed on the classification recognition results corresponding to each data block to obtain the target detection result corresponding to the software to be detected.

[0262] Remember each data block B i The corresponding position weight is w i , then the formula for obtaining the final target detection result can be:

[0263]

[0264] Among them, I(c i =pm) is an indicator function, which takes the value of 1 when the data block is classified into the preset target category, and 0 otherwise.

[0265] Furthermore, a corresponding feature weight can be set for each data block based on the data distribution characteristics corresponding to each data block, so that the feature weight corresponding to the data block with certain special characteristics is increased, thereby further improving the accuracy of the target detection result.

[0266] In a possible implementation, in order to consider that consecutive data blocks of potential preset target categories may increase the possibility that the software to be detected is determined to be a target category, the server may introduce a number value Q that continuously indicates that the corresponding data blocks are of the preset target category.

[0267] Specifically, the server performs weighted fusion processing on the classification and recognition results corresponding to each data block based on the position weight corresponding to each acquired data block to obtain the corresponding intermediate detection result; then, based on the classification and recognition result, the server obtains the number value Q of the classification and recognition results that continuously characterize the corresponding data blocks as the preset target category in all data blocks.

[0268] For example, assume that there are 5 data blocks, which are identified as the classification recognition results of the preset target category. Based on the position information of the data blocks, they are: yes, no, yes, yes, no; then, for this classification recognition result, the number value Q=2 that continuously represents the corresponding data blocks as the preset target category.

[0269] In this way, the server can correct the intermediate detection result based on the quantity value and obtain the target detection result corresponding to the software to be detected.

[0270] For example, in this method, the target detection result S is obtained as follows:

[0271]

[0272] Where a is a positive number used to adjust the influence of data blocks of consecutive potential preset target categories on the final suspiciousness level.

[0273] Finally, if S exceeds a predetermined threshold θ, the software to be detected is determined to be target type software.

[0274] This solution performs targeted software type detection by segmenting the program files of the software being tested and performing byte-level analysis of the file contents. Because this solution is independent of file type, it is applicable to all file types and can be tested without prior knowledge of the file format, greatly enhancing its versatility and practicality.

[0275] In addition, this solution classifies and identifies data blocks based on the distribution of various sets of continuous byte sequences in the data blocks divided by program files. It can capture the inherent differences between the target software and normal files in the underlying byte-level content, thereby effectively detecting target software even when it appears for the first time or has been disguised.

[0276] Based on the same inventive concept, an embodiment of the present application also provides a software detection device.

[0277] See also Figure 10 , which is a structural diagram of a software detection device provided in an embodiment of the present application. The device can be the above-mentioned server or a chip or integrated circuit in the device, etc. The device includes modules / units / technical means for executing the method executed by the server in the above-mentioned method embodiment.

[0278] Exemplarily, the apparatus 1000 includes:

[0279] The acquisition module 1001 is configured to acquire the software to be detected and divide the program file of the software to be detected into at least two data blocks; wherein each data block has the same byte size;

[0280] Identification module 1002 is configured to perform the following operations for each of the at least two data blocks: obtaining a distribution of at least one continuous byte sequence set in the data block, wherein the continuous byte sequence is any continuous byte of a fixed byte length in the data block, and each continuous byte sequence set has a different byte length; performing classification and identification processing on the data block based on the obtained distribution of each continuous byte sequence set, and obtaining a classification and identification result corresponding to the data block; wherein the classification and identification result represents a probability that the data block belongs to a preset target category;

[0281] The processing module 1003 is used to perform weighted fusion on the at least two classification recognition results obtained based on the classification recognition results corresponding to the at least two data blocks obtained, combined with the position information of each data block in the program file, to obtain the target detection result corresponding to the software to be detected.

[0282] Optionally, the identification module 1002 is configured to obtain at least one set of continuous byte sequences in a data block, and when determining the distribution of the set of continuous byte sequences in the data block, is specifically configured to:

[0283] Extracting at least one continuous byte sequence set from all bytes contained in the one data block; wherein the byte length corresponding to each continuous byte sequence set is one of the preset byte length sets;

[0284] For the at least one continuous byte sequence set, respectively obtaining the occurrence frequency of each continuous byte sequence in each continuous byte sequence set in the corresponding continuous byte sequence set;

[0285] Based on the obtained occurrence frequency corresponding to each continuous sequence of bytes, the distribution of each continuous byte sequence set in the data block is extracted.

[0286] Optionally, the identification module 1002 is configured to obtain, for the at least one continuous byte sequence set, respectively the occurrence frequency of each continuous byte sequence in each continuous byte sequence set, specifically for:

[0287] For each of the at least one continuous byte sequence set, perform the following operations respectively:

[0288] Obtaining the number of occurrences of each continuous byte sequence in a continuous byte sequence set;

[0289] Based on the number of occurrences and the total number of continuous byte sequences in the continuous byte sequence set, the occurrence frequency of each continuous byte sequence in the continuous byte sequence set is obtained.

[0290] Optionally, the distribution situation includes at least one or more of the following:

[0291] A Simpson index determined based on the frequency of occurrence of each of the consecutive sequence bytes;

[0292] The Canberra distance is determined based on the frequency of occurrence of two consecutive sequence bytes;

[0293] Minkowski distance determined based on the frequency of occurrence of two adjacent consecutive sequence bytes;

[0294] Manhattan distance determined based on the frequency of occurrence of two consecutive sequence bytes;

[0295] The Chebyshev distance is determined based on the frequency of occurrence of two adjacent consecutive sequence bytes;

[0296] Bray-Curtis distance based on the frequency of occurrence of two consecutive sequence bytes;

[0297] Angular separation determined based on the frequency of occurrence of two adjacent consecutive sequence bytes;

[0298] A correlation coefficient determined based on the occurrence frequency corresponding to each continuous sequence of bytes and the average occurrence probability of the corresponding continuous sequence of bytes;

[0299] The total variation determined based on the frequency of occurrence of two consecutive sequence bytes;

[0300] An information entropy determined based on the frequency of occurrence of each consecutive sequence of bytes;

[0301] Kuhler-Leibler divergence based on the frequency of occurrence of two consecutive sequence bytes;

[0302] Jensen-Shannon divergence based on the frequency of occurrence of two consecutive sequence bytes;

[0303] The Ito-Saito divergence is determined based on the frequency of occurrence of two adjacent consecutive sequence bytes.

[0304] Optionally, the identification module 1002 is configured to perform classification and identification processing on the data block based on the distribution corresponding to each continuous byte sequence set obtained, and when obtaining the classification and identification result corresponding to the data block, specifically to:

[0305] Performing classification and recognition processing on the data block based on the obtained distribution of each continuous byte sequence set using the trained classification and recognition model to obtain a classification and recognition result corresponding to the data block;

[0306] The classification recognition model includes at least two sub-classification identifiers, and each sub-classification identifier is used to perform classification recognition processing on a preset classification in the preset classification set.

[0307] Optionally, the recognition module 1002 is configured to perform classification and recognition processing on the data block based on the distribution corresponding to each continuous byte sequence set obtained using a trained classification and recognition model, and to obtain a classification and recognition result corresponding to the data block, specifically for:

[0308] Combining the obtained distribution conditions corresponding to each set of continuous byte sequences into the data distribution characteristics corresponding to the one data block;

[0309] Performing classification and recognition processing on the data distribution feature by each of the at least two sub-classification identifiers, and obtaining a sub-recognition result corresponding to each sub-classification identifier;

[0310] Based on the weight coefficient preset for each sub-classification identifier, the at least two sub-recognition results obtained are weightedly fused to obtain a classification recognition result corresponding to the one data block.

[0311] Optionally, the processing module 1003 is configured to perform weighted fusion on the at least two classification and recognition results obtained based on the classification and recognition results corresponding to the at least two data blocks obtained, combined with the position information of each data block in the program file, to obtain the target detection result corresponding to the software to be detected, specifically for:

[0312] Based on the position information of each data block in the program file, obtaining a position weight corresponding to each data block; the position weight is used to represent the importance of the corresponding data block in the program file;

[0313] Based on the acquired position weight corresponding to each data block, weighted fusion processing is performed on the classification recognition results corresponding to each data block to obtain the target detection result corresponding to the software to be detected.

[0314] Optionally, the processing module 1003 is configured to perform weighted fusion processing on the classification recognition results corresponding to each data block based on the acquired position weight corresponding to each data block, and to obtain the target detection result corresponding to the software to be detected, specifically for:

[0315] Based on the acquired position weights corresponding to each data block, weighted fusion processing is performed on the classification recognition results corresponding to each data block to obtain a corresponding intermediate detection result;

[0316] Based on the classification and recognition results, obtaining, in all data blocks, a number of data blocks continuously characterized by the classification and recognition results as belonging to the preset target category;

[0317] Based on the quantity value, the intermediate detection result is corrected to obtain the target detection result corresponding to the software to be detected.

[0318] As an example, Figure 10 The apparatus discussed can be used to perform Figure 2 Therefore, for the functions that can be realized by each functional module of the device, please refer to Figure 2 The description of the illustrated embodiment will not be repeated here.

[0319] It should be noted that although several modules or submodules of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more units described above may be embodied in one module. Conversely, the features and functions of one module described above may be further divided and embodied by multiple modules.

[0320] As a possible product form of the above device, see Figure 11 , an embodiment of the present application further provides an electronic device 1100, including:

[0321] At least one processor 1101; and a communication interface 1103 communicatively connected to the at least one processor 1101; the at least one processor 1101 executes instructions stored in the memory 1102, so that the electronic device 1100 executes the method steps performed by any device in the above method embodiment through the communication interface 1103.

[0322] Optionally, the memory 1102 is located outside the electronic device 1100 .

[0323] Optionally, the electronic device 1100 includes the memory 1102, the memory 1102 is connected to the at least one processor 1101, and the memory 1102 stores instructions that can be executed by the at least one processor 1101. Figure 11 The dashed lines indicate that the memory 1102 is optional for the electronic device 1100 .

[0324] The processor 1101 and the memory 1102 may be coupled via an interface circuit or may be integrated together, which is not limited here.

[0325] The specific connection medium between the processor 1101, the memory 1102 and the communication interface 1103 is not limited in the embodiment of the present application. Figure 11 The processor 1101, the memory 1102 and the communication interface 1103 are connected via a bus 1104. Figure 11 The connections between the other components are shown in bold lines, which are only for illustration and are not intended to be limiting. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, Figure 11 The bus is represented by only one thick line, but this does not mean that there is only one bus or one type of bus. It should be understood that the processor mentioned in the embodiments of the present application can be implemented by hardware or software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented by software, the processor can be a general-purpose processor that is implemented by reading software code stored in a memory.

[0326] Exemplarily, the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0327] It should be understood that the memory mentioned in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0328] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, the memory (storage module) can be integrated into the processor.

[0329] It should be noted that the memory described herein is intended to include, but not be limited to, these and any other suitable types of memory.

[0330] As another possible product form, an embodiment of the present application also provides a computer-readable storage medium, which is used to store instructions. When the instructions are executed, the computer executes the method steps performed by any device in the above method example.

[0331] As another possible product form, an embodiment of the present application also provides a computer program product containing instructions, wherein the computer program product stores instructions that, when executed on a computer, enable the computer to execute the method steps performed by any device in the above method embodiment.

[0332] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0333] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0334] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0335] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0336] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A method for software detection, characterized in that: The method comprises: Obtain the software to be detected, and divide the program file of the software to be detected into at least two data blocks; wherein the byte size of each data block is the same; For each of the at least two data blocks, perform the following operations respectively: Obtaining a distribution of at least one continuous byte sequence set in a data block in the data block; wherein the continuous byte sequence is: any continuous bytes with a fixed byte length in the data block, and each continuous byte sequence set corresponds to a different byte length; Based on the distribution of each continuous byte sequence set obtained, the data block is classified and identified to obtain a classification and identification result corresponding to the data block; wherein the classification and identification result represents: the probability that the data block is a preset target category; Based on the classification and recognition results corresponding to the at least two data blocks obtained, combined with the position information of each data block in the program file, the at least two classification and recognition results obtained are weightedly fused to obtain the target detection result corresponding to the software to be detected.

2. The method according to claim 1, characterized in that The obtaining of at least one continuous byte sequence set in a data block and distribution of the set in the data block includes: Extracting at least one continuous byte sequence set from all bytes contained in the one data block; wherein the byte length corresponding to each continuous byte sequence set is one of the preset byte length sets; For the at least one continuous byte sequence set, respectively obtain the occurrence frequency of each continuous byte sequence in each continuous byte sequence set in the corresponding continuous byte sequence set; Based on the obtained occurrence frequency corresponding to each continuous sequence of bytes, the distribution of each continuous byte sequence set in the data block is extracted.

3. The method according to claim 2, characterized in that The acquiring, for the at least one continuous byte sequence set, respectively the occurrence frequency of each continuous byte sequence in each continuous byte sequence set in the corresponding continuous byte sequence set comprises: For each of the at least one continuous byte sequence set, perform the following operations respectively: Obtaining the number of occurrences of each continuous byte sequence in a continuous byte sequence set; Based on the number of occurrences and the total number of continuous byte sequences in the continuous byte sequence set, the occurrence frequency of each continuous byte sequence in the continuous byte sequence set is obtained.

4. The method according to claim 2, characterized in that The distribution conditions include at least one or more of the following: A Simpsons index determined based on the frequency of occurrence of each of the consecutive sequence bytes; The Canberra distance is determined based on the frequency of occurrence of two adjacent consecutive sequence bytes; The Minkowski distance is determined based on the frequency of occurrence of two adjacent consecutive sequence bytes; The Manhattan distance is determined based on the frequency of occurrence of two consecutive sequence bytes; The Chebyshev distance is determined based on the frequency of occurrence of two adjacent consecutive sequence bytes; Bray-Curtis distance determined based on the frequency of occurrence of two adjacent consecutive sequence bytes; Angular separation determined based on the frequency of occurrence of two adjacent consecutive sequence bytes; A correlation coefficient determined based on the occurrence frequency corresponding to each continuous sequence of bytes and the average occurrence probability of the corresponding continuous sequence of bytes; The total variation determined based on the frequency of occurrence of two consecutive sequence bytes; Information entropy determined based on the frequency of occurrence of each continuous sequence of bytes; The Kuhler-Leibler divergence is determined based on the frequency of occurrence of two consecutive sequence bytes; The Jensen-Shannon divergence is determined based on the frequency of occurrence of two consecutive sequence bytes; The Ito-Saito divergence is determined based on the frequency of occurrence of two adjacent consecutive sequence bytes.

5. The method according to any one of claims 1 to 4, characterized in that: The step of performing classification and identification processing on the data block based on the obtained distribution of each continuous byte sequence set to obtain a classification and identification result corresponding to the data block includes: Using the trained classification and recognition model, based on the distribution of each continuous byte sequence set obtained, the data block is classified and recognized to obtain a classification and recognition result corresponding to the data block; The classification recognition model includes at least two sub-classification identifiers, and each sub-classification identifier is used to perform classification recognition processing on a preset classification in the preset classification set.

6. The method according to claim 5, characterized in that The method of performing classification and recognition processing on the data block based on the distribution of each continuous byte sequence set obtained by using the trained classification and recognition model to obtain a classification and recognition result corresponding to the data block includes: Combining the obtained distribution conditions corresponding to each continuous byte sequence set into a data distribution feature corresponding to the one data block; By using each of the at least two sub-classification identifiers, respectively performing classification and identification processing on the data distribution feature, and obtaining a sub-identification result corresponding to each sub-classification identifier; Based on the weight coefficient preset for each sub-classification identifier, the at least two sub-recognition results obtained are weighted and fused to obtain the classification recognition result corresponding to the one data block.

7. The method according to any one of claims 1 to 4, characterized in that: The method of performing weighted fusion on the at least two classification recognition results obtained based on the classification recognition results corresponding to the at least two data blocks respectively obtained and combining the position information of each data block in the program file to obtain the target detection result corresponding to the software to be detected includes: Based on the position information of each data block in the program file, obtaining the position weight corresponding to each data block; the position weight is used to represent the importance of the corresponding data block in the program file; Based on the acquired position weights corresponding to each of the data blocks, weighted fusion processing is performed on the classification and recognition results corresponding to each of the data blocks to obtain the target detection results corresponding to the software to be detected.

8. The method according to claim 7, characterized in that The step of performing weighted fusion processing on the classification recognition results corresponding to each data block based on the acquired position weight corresponding to each data block to obtain the target detection result corresponding to the software to be detected includes: Based on the acquired position weights corresponding to each of the data blocks, weighted fusion processing is performed on the classification recognition results corresponding to each of the data blocks to obtain corresponding intermediate detection results; Based on the classification and recognition results, obtaining, among all the data blocks, a number value of the data blocks that are continuously characterized by the classification and recognition results as belonging to the preset target category; Based on the quantity value, the intermediate detection result is corrected to obtain the target detection result corresponding to the software to be detected.

9. A software detection device, characterized in that: The device comprises: The acquisition module is used to acquire the software to be detected and divide the program file of the software to be detected into at least two data blocks; wherein the byte size of each data block is the same; The identification module is used to perform the following operations for each of the at least two data blocks: obtaining the distribution of at least one continuous byte sequence set in a data block in the data block; wherein the continuous byte sequence is any continuous byte with a fixed byte length in the data block, and each continuous byte sequence set has a different byte length; based on the obtained distribution corresponding to each continuous byte sequence set, classifying and identifying the data block, and obtaining a classification identification result corresponding to the data block; wherein the classification identification result represents the probability that the data block is a preset target category; The processing module is used to perform weighted fusion on the at least two classification recognition results obtained based on the classification recognition results corresponding to the at least two data blocks obtained, combined with the position information of each data block in the program file, to obtain the target detection result corresponding to the software to be detected.

10. An electronic device, characterized in that: include: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the at least one processor executes the method as described in any one of claims 1 to 8 by executing the instructions stored in the memory.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store instructions, and when the instructions are executed, the method according to any one of claims 1 to 8 is implemented.

12. A computer program product comprising instructions, characterized in that The computer program product stores instructions, and when the computer program product is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 8.

Images