A method for determining malicious URLs

By analyzing log data and using the Euro-style distance algorithm to match feature values ​​from the URL sample library, combined with sandboxing technology, the problem of inaccurate malicious URL judgment in the existing technology is solved, and the accuracy of malicious URL detection is improved.

CN116432178BActive Publication Date: 2024-11-08HUANENG INFORMATION TECH CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310331694.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-03-30
Publication Date
2024-11-08
Estimated Expiration
2043-03-30

AI Technical Summary

Technical Problem

The method of determining malicious URLs in the prior art is based on website links and lacks accuracy. In particular, malicious websites may use inconspicuous URLs or add meaningless strings, resulting in difficulty in detection.

Method used

By reading and parsing the log data to be detected, extracting the characteristic values ​​of the URL, and matching them from the preset URL sample library according to the characteristic similarity, using the European distance algorithm to determine the URL type, and combining sandbox technology to perform malicious URL detection, reducing the Dependence on URL links.

Benefits of technology

It improves the accuracy of malicious URL judgment, reduces the interference of URL links on detection, and achieves higher accuracy of detection results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116432178B_ABST
    Figure CN116432178B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of data processing technology, and in particular to a method for determining malicious URLs. The method comprises: reading and parsing log data to be detected to obtain a URL to be detected; extracting feature values ​​of multiple features of the URL to be detected; matching multiple URL samples from a preset URL sample library in descending order of feature similarity according to the feature values ​​of the multiple features; counting the number of URL samples of normal URL type and malicious URL type in multiple URL samples, and taking the URL type with the largest number of URL samples as the URL type of the URL to be detected, and when the URL type of the URL to be detected is a malicious URL type, performing malicious URL detection on the log data to be detected by using sandbox technology, and obtaining a detection result. The present invention performs targeted matching according to multiple types of URL samples, and determines whether the URL to be detected is a malicious URL type according to similarity, thereby changing the traditional form of determination based on website links and greatly improving the accuracy of the detection result.
Need to check novelty before this filing date? Find Prior Art

Description

Background Art

[0002] Malicious URLs refer to malicious websites. Malicious websites are used as a collective noun to refer to illegal websites that contain viruses, worms, and Trojan horses that intentionally perform malicious tasks on computer systems. These websites usually have one thing in common. They are usually in the form of some kind of web page that allows people to browse the page content normally, while illegally obtaining various data in the computer.

[0003] However, in the prior art, the determination method for malicious URLs is based on website links. Usually, malicious websites will induce users to visit by linking to other websites. The first sign of linking to a malicious website is usually that the URL is incomplete or unclear, and the complete URL is not used. However, the method of determining malicious URLs based on website links is not accurate, because malicious websites may use unclear URLs, or add meaningless strings to URLs with malicious website link characteristics, resulting in interference during detection and inability to accurately determine. Therefore, how to provide a malicious URL determination method is a technical problem that technical personnel in this field urgently need to solve. Summary of the invention

[0004] The purpose of the present invention is to provide a malicious URL determination method. The present invention performs targeted matching based on various types of URL samples and determines whether the URL to be detected is a malicious URL type based on similarity, thereby changing the traditional form of determination based on website links and greatly improving the accuracy of the detection results.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A method for determining a malicious URL, comprising:

[0007] Read and parse the log data to be detected to obtain the URL to be detected;

[0008] Extracting feature values ​​of multiple features of the URL to be detected;

[0009] According to the feature values ​​of the plurality of features, a plurality of URL samples are matched from a preset URL sample library in descending order of feature similarity; wherein,

[0010] The preset URL sample library includes URL samples of multiple URL types, the multiple URL types include normal URL types and malicious URL types, and the feature similarity is determined based on the Euclidean distance;

[0011] Count the number of URL samples of the normal URL type and the malicious URL type in the multiple URL samples, and use the URL type with the largest number of the URL samples as the URL type of the URL to be detected; when the URL type of the URL to be detected is the malicious URL type, perform malicious URL detection on the log data to be detected through sandbox technology, and obtain a detection result.

[0012] In some embodiments of the present application, the matching of multiple URL samples from a preset URL sample library in descending order of feature similarity according to the feature values ​​of the multiple features includes:

[0013] In a multidimensional vector space with the multiple features as dimensions, calculating the Euclidean distance between the URL to be detected and the multiple URL samples in the preset URL sample library;

[0014] The URL samples whose Euclidean distance to the URL to be detected is less than a preset standard distance are taken as the multiple URL samples.

[0015] In some embodiments of the present application, the matching of multiple URL samples from a preset URL sample library in descending order of feature similarity according to the feature values ​​of the multiple features further includes:

[0016] In a multidimensional vector space with the multiple features as dimensions, calculating the Euclidean distance between the URL to be detected and the multiple URL samples in the preset URL sample library;

[0017] A preset number of URL samples are selected from the preset URL sample library in descending order of the distance value N of the Euclidean distance as the multiple URL samples.

[0018] In some embodiments of the present application, a preset Euclidean distance value matrix T0 and a preset selected URL sample quantity matrix A are pre-set. For the preset selected URL sample quantity matrix A, A(A1, A2, A3, A4) is set, wherein A1 is the first preset selected URL sample quantity, A2 is the second preset selected URL sample quantity, A3 is the third preset selected URL sample quantity, A4 is the fourth preset selected URL sample quantity, and A1<A2<A3<A4; for the preset Euclidean distance value matrix T0, T0(T01, T02, T03, T04) is set, wherein T01 is the first preset Euclidean distance value, T02 is the second preset Euclidean distance value, T03 is the third preset Euclidean distance value, T04 is the fourth preset Euclidean distance value, and T01<T02<T03<T04;

[0019] According to the relationship between N and the preset Euclidean distance value matrix T0, a corresponding number of selected URL samples is selected as the multiple URL samples;

[0020] When N<T01, the first preset number of selected URL samples A1 is selected as the multiple URL samples;

[0021] When T01≤N<T02, the second preset number of selected URL samples A2 is selected as the plurality of URL samples;

[0022] When T02≤N<T03, the third preset number of selected URL samples A3 is selected as the multiple URL samples;

[0023] When T03≤N<T04, the fourth preset number A4 of selected URL samples is selected as the plurality of URL samples.

[0024] In some embodiments of the present application, the malicious URL detection is performed on the log data to be detected by using sandbox technology, and the detection result is obtained, including:

[0025] Obtain training samples, and train the static sandbox and the dynamic sandbox using sandbox technology according to the training samples;

[0026] Performing static analysis based on the static data in the log data to be detected and the static sandbox, and obtaining a first analysis result of the URL to be detected;

[0027] Performing dynamic analysis according to the dynamic data in the log data to be detected and the dynamic sandbox, and obtaining a second analysis result of the URL to be detected;

[0028] The detection result is obtained according to the first analysis result of the URL to be detected and the second analysis result of the URL to be detected.

[0029] In some embodiments of the present application, when the URL to be detected is determined to be a malicious URL based on the detection result of the log data to be detected, the alarm information corresponding to the detection result is issued in real time based on the destination IP and asset information corresponding to the URL to be detected.

[0030] In some embodiments of the present application, the performing malicious URL detection on the log data to be detected by using sandbox technology and obtaining the detection result also includes:

[0031] When the URL to be detected is determined to be a malicious URL according to the detection result of the log data to be detected, a real-time alarm is issued according to the detection result through a preset processing terminal; wherein,

[0032] An asset information table and an alarm information table are set in the preset processing terminal. The asset information table includes the source destination IP, the source destination port and the detected URL. The alarm information table includes the source destination IP, the detected URL and the malicious type.

[0033] In some embodiments of the present application, when the URL type of the URL to be detected is not less than two of the malicious URL types, the distance value of the Euclidean distance is corrected.

[0034] In some embodiments of the present application, the real-time alarming according to the detection result by a preset processing terminal further includes:

[0035] The source destination IP, the detected URL and the malicious type are saved.

[0036] The present invention provides a malicious URL determination method, which has the following beneficial effects compared with the prior art:

[0037] The present invention obtains the URL to be detected by reading and parsing the log data to be detected, extracts the feature values ​​of multiple features of the URL to be detected, and matches multiple URL samples from a preset URL sample library in the order of feature similarity from large to small according to the feature values ​​of the multiple features; determines whether the URL to be detected is a malicious URL type according to the matching samples, and uses the form of matching of multiple samples without relying on the need to detect the website link every time, thereby reducing the interference of the website link on the detection and judgment, and improving the accuracy of malicious URL judgment. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a flow chart of a method for determining a malicious URL in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0040] In the description of the present application, it should be understood that the terms "center", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0041] The terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, unless otherwise specified, "plurality" means two or more.

[0042] In the description of this application, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be a communication between the inner sides of two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0043] In the prior art, the determination method for malicious URLs is based on website links. Usually, malicious websites will induce users to visit by linking to other websites. The first sign of linking to a malicious website is usually that the URL is incomplete or unclear, and the complete URL is not used. However, the method of determining malicious URLs based on website links is not accurate, because malicious websites may use unclear URLs, or add meaningless strings to URLs with malicious website link characteristics, resulting in interference during detection and inability to accurately determine the URL.

[0044] Therefore, the present invention provides a malicious URL determination method, which performs targeted matching based on multiple types of URL samples and determines whether the URL to be detected is a malicious URL type based on similarity, thereby changing the traditional form of determination based on website links and greatly improving the accuracy of the detection results.

[0045] See also Figure 1 As shown, the disclosed embodiment of the present invention provides a malicious URL determination method, comprising:

[0046] Read and parse the log data to be detected to obtain the URL to be detected;

[0047] Extracting feature values ​​of multiple features of the URL to be detected;

[0048] According to the feature values ​​of multiple features, multiple URL samples are matched from a preset URL sample library in descending order of feature similarity; wherein,

[0049] The preset URL sample library includes URL samples of multiple URL types, including normal URL types and malicious URL types. The feature similarity is determined based on the Euclidean distance.

[0050] The number of URL samples of normal URL type and malicious URL type in multiple URL samples is counted, and the URL type with the largest number of URL samples is used as the URL type of the URL to be detected. When the URL type of the URL to be detected is a malicious URL type, malicious URL detection is performed on the log data to be detected through sandbox technology, and the detection result is obtained.

[0051] In a specific embodiment of the present application, multiple URL samples are matched from a preset URL sample library in descending order of feature similarity according to feature values ​​of multiple features, including:

[0052] In a multidimensional vector space with multiple features as dimensions, calculating the Euclidean distance between the URL to be detected and multiple URL samples in a preset URL sample library;

[0053] The URL samples whose Euclidean distance to the URL to be detected is less than a preset standard distance are taken as multiple URL samples.

[0054] In a specific embodiment of the present application, according to the feature values ​​of the multiple features, multiple URL samples are matched from a preset URL sample library in descending order of feature similarity, and further comprising:

[0055] In a multidimensional vector space with multiple features as dimensions, calculating the Euclidean distance between the URL to be detected and multiple URL samples in a preset URL sample library;

[0056] A preset number of URL samples are selected from a preset URL sample library in descending order of the distance value N of the Euclidean distance as a plurality of URL samples.

[0057] In a specific embodiment of the present application, a preset Euclidean distance value matrix T0 and a preset selected URL sample quantity matrix A are pre-set. For the preset selected URL sample quantity matrix A, A(A1, A2, A3, A4) is set, wherein A1 is the first preset selected URL sample quantity, A2 is the second preset selected URL sample quantity, A3 is the third preset selected URL sample quantity, A4 is the fourth preset selected URL sample quantity, and A1<A2<A3<A4; for the preset Euclidean distance value matrix T0, T0(T01, T02, T03, T04) is set, wherein T01 is the first preset Euclidean distance value, T02 is the second preset Euclidean distance value, T03 is the third preset Euclidean distance value, T04 is the fourth preset Euclidean distance value, and T01<T02<T03<T04;

[0058] According to the relationship between N and the preset Euclidean distance value matrix T0, a corresponding number of selected URL samples is selected as a plurality of URL samples;

[0059] When N<T01, the first preset number of selected URL samples A1 is selected as the multiple URL samples;

[0060] When T01≤N<T02, a second preset number of selected URL samples A2 is selected as the plurality of URL samples;

[0061] When T02≤N<T03, a third preset number of selected URL samples A3 is selected as the plurality of URL samples;

[0062] When T03≤N<T04, the fourth preset number of selected URL samples A4 is selected as the plurality of URL samples.

[0063] In a specific embodiment of the present application, malicious URL detection is performed on the log data to be detected by using sandbox technology, and the detection results are obtained, including:

[0064] Obtain training samples, and train the static sandbox and the dynamic sandbox using the sandbox technology according to the training samples;

[0065] Perform static analysis based on static data and static sandbox in the log data to be detected, and obtain a first analysis result of the URL to be detected;

[0066] Perform dynamic analysis based on dynamic data and dynamic sandbox in the log data to be detected, and obtain a second analysis result of the URL to be detected;

[0067] A detection result is obtained according to the first analysis result of the URL to be detected and the second analysis result of the URL to be detected.

[0068] In a specific embodiment of the present application, when the URL to be detected is determined to be a malicious URL based on the detection result of the log data to be detected, the alarm information corresponding to the detection result is issued in real time based on the destination IP and asset information corresponding to the URL to be detected.

[0069] In a specific embodiment of the present application, malicious URL detection is performed on the log data to be detected by using sandbox technology, and the detection result is obtained, which also includes:

[0070] When the URL to be detected is determined to be a malicious URL according to the detection result of the log data to be detected, a real-time alarm is issued according to the detection result through the preset processing terminal; wherein,

[0071] An asset information table and an alarm information table are set in the preset processing terminal. The asset information table includes source and destination IP, source and destination port and detected URL, and the alarm information table includes source and destination IP, detected URL and malicious type.

[0072] In a specific embodiment of the present application, when the URL type of the URL to be detected is no less than two malicious URL types, the distance value of the Euclidean distance is corrected.

[0073] In a specific embodiment of the present application, a real-time alarm is issued according to the detection result by a preset processing terminal, and further includes:

[0074] Save the source and destination IP, detected URLs, and malicious types.

[0075] In summary, the present invention obtains the URL to be detected by reading and parsing the log data to be detected, extracts the feature values ​​of multiple features of the URL to be detected, and matches multiple URL samples from a preset URL sample library in the order of feature similarity from large to small according to the feature values ​​of the multiple features; determines whether the URL to be detected is a malicious URL type according to the matching samples, and matches multiple samples in the form of matching, and combines the Euclidean distance algorithm, based on the absolute value of the distance in the multidimensional space, determines the characteristics of similarity, does not rely on the traditional method of needing to detect the URL link every time, reduces the interference of the URL link to the detection and judgment, and improves the accuracy of the malicious URL judgment.

[0076] The above is only an example of implementation of the present invention, but it cannot be used to limit the scope of the present invention. Any structural changes made according to the present invention should be regarded as falling within the protection scope of the present invention and subject to restrictions as long as they do not lose the essence of the present invention.

[0077] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process and related instructions of the system described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0078] It should be noted that the system provided in the above embodiment is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the modules or steps in the embodiments of the present invention can be decomposed or combined. For example, the modules in the above embodiments can be combined into one module, or further divided into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the modules or steps, and are not regarded as improper limitations of the present invention.

[0079] Those skilled in the art should be able to appreciate that the modules and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of the two, and the programs corresponding to the software modules and method steps can be placed in random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the technical field. In order to clearly illustrate the interchangeability of electronic hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are performed in electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0080] The term "comprise" or any other similar term is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus / device that includes a list of elements includes not only those elements but also other elements not expressly listed, or also includes elements inherent to such process, method, article, or apparatus / device.

[0081] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

[0082] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention.

Claims

1. A method for determining malicious URLs, characterized in that: include: Read and parse the log data to be detected to obtain the URL to be detected; Extracting feature values ​​of multiple features of the URL to be detected; According to the feature values ​​of the plurality of features, a plurality of URL samples are matched from a preset URL sample library in descending order of feature similarity; wherein, The preset URL sample library includes URL samples of multiple URL types, the multiple URL types include normal URL types and malicious URL types, and the feature similarity is determined based on the Euclidean distance; Counting the number of URL samples of the normal URL type and the malicious URL type in the multiple URL samples, and taking the URL type with the largest number of the URL samples as the URL type of the URL to be detected; when the URL type of the URL to be detected is the malicious URL type, performing malicious URL detection on the log data to be detected by using sandbox technology, and obtaining a detection result; The malicious URL detection is performed on the log data to be detected by using the sandbox technology, and the detection result is obtained, including: Obtain training samples, and train the static sandbox and the dynamic sandbox using sandbox technology according to the training samples; Performing static analysis based on the static data in the log data to be detected and the static sandbox, and obtaining a first analysis result of the URL to be detected; Performing dynamic analysis according to the dynamic data in the log data to be detected and the dynamic sandbox, and obtaining a second analysis result of the URL to be detected; Obtaining the detection result according to the first analysis result of the URL to be detected and the second analysis result of the URL to be detected; When the URL to be detected is determined to be a malicious URL according to the detection result of the log data to be detected, the alarm information corresponding to the detection result is issued in real time according to the destination IP and asset information corresponding to the URL to be detected; The method of performing malicious URL detection on the log data to be detected by using sandbox technology and obtaining the detection result also includes: When the URL to be detected is determined to be a malicious URL according to the detection result of the log data to be detected, a real-time alarm is issued according to the detection result through a preset processing terminal; wherein, An asset information table and an alarm information table are set in the preset processing terminal. The asset information table includes the source destination IP, the source destination port and the detected URL. The alarm information table includes the source destination IP, the detected URL and the malicious type.

2. A malicious URL determination method according to claim 1, characterized in that: The matching of multiple URL samples from a preset URL sample library according to the feature values ​​of the multiple features in descending order of feature similarity includes: In a multidimensional vector space with the multiple features as dimensions, calculating the Euclidean distance between the URL to be detected and the multiple URL samples in the preset URL sample library; The URL samples whose Euclidean distance to the URL to be detected is less than a preset standard distance are taken as the multiple URL samples.

3. A malicious URL determination method according to claim 2, characterized in that: The matching of multiple URL samples from a preset URL sample library in descending order of feature similarity according to the feature values ​​of the multiple features also includes: In a multidimensional vector space with the multiple features as dimensions, calculating the Euclidean distance between the URL to be detected and the multiple URL samples in the preset URL sample library; A preset number of URL samples are selected from the preset URL sample library in descending order of the distance value N of the Euclidean distance as the multiple URL samples.

4. A malicious URL determination method according to claim 3, characterized in that: A preset Euclidean distance value matrix T0 and a preset selected URL sample quantity matrix A are pre-set. For the preset selected URL sample quantity matrix A, A(A1, A2, A3, A4) is set, wherein A1 is the first preset selected URL sample quantity, A2 is the second preset selected URL sample quantity, A3 is the third preset selected URL sample quantity, A4 is the fourth preset selected URL sample quantity, and A1<A2<A3<A4; for the preset Euclidean distance value matrix T0, T0(T01, T02, T03, T04) is set, wherein T01 is the first preset Euclidean distance value, T02 is the second preset Euclidean distance value, T03 is the third preset Euclidean distance value, T04 is the fourth preset Euclidean distance value, and T01<T02<T03<T04; According to the relationship between N and the preset Euclidean distance value matrix T0, a corresponding number of selected URL samples is selected as the multiple URL samples; When N<T01, the first preset number of selected URL samples A1 is selected as the multiple URL samples; When T01≤N<T02, the second preset number of selected URL samples A2 is selected as the plurality of URL samples; When T02≤N<T03, the third preset number of selected URL samples A3 is selected as the multiple URL samples; When T03≤N<T04, the fourth preset number A4 of selected URL samples is selected as the plurality of URL samples.

5. A malicious URL determination method according to claim 4, characterized in that: When the URL type of the URL to be detected is not less than two of the malicious URL types, the distance value of the Euclidean distance is corrected.

6. A malicious URL determination method according to claim 1, characterized in that: The real-time alarm is issued according to the detection result by the preset processing terminal, and further includes: The source destination IP, the detected URL and the malicious type are saved.