Data interaction path association reasoning method and system based on multi-source heterogeneous log

Through the data interaction path association reasoning method of multi-source heterogeneous logs, the problem that HTTP requests and database requests cannot be associated in traditional data traceability is solved, and the accuracy of data traceability and data security supervision capabilities are improved.

CN118890168APending Publication Date: 2024-11-01GLOBAL ENERGY INTERCONNECTION RES INST CO LTD +3
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410855487.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-28
Publication Date
2024-11-01

AI Technical Summary

Technical Problem

Traditional data traceability devices cannot decrypt encrypted data traffic with high security levels, and cannot directly forward data records with privacy information, resulting in the inability to associate HTTP request analysis with database request analysis, and the accuracy of data traceability is poor.

Method used

Through the data interaction path association inference method based on multi-source heterogeneous logs, an HTTP interaction log array list and a database interaction log array list are generated, a time bitmap is constructed and an access path pattern array list is generated, and an association inference is performed to determine the access path pattern list and Access the associative array list between table schema lists to realize the time association between HTTP requests and database requests.

Benefits of technology

It improves the accuracy of data traceability, further strengthens the ability to supervise data security, and obtains the flow paths of data on HTTP servers and database servers through time correlation to obtain more comprehensive data information.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118890168A_ABST
    Figure CN118890168A_ABST
Patent Text Reader

Abstract

The invention provides a data interaction path association reasoning method and system based on a multi-source heterogeneous log, and belongs to the technical field of information security, and the method comprises the steps: generating an HTTP interaction log array list based on log data of an HTTP server, and generating an access path mode array list according to a time bitmap and the HTTP interaction log array list; generating a database interaction log array list based on log data of the database, and generating an access table mode array list according to the time bitmap and the database interaction log array list; and performing association reasoning on the access path mode array list and the access table mode array list, and determining an association array list between the access path mode list and the access table mode list. According to the method, HTTP request analysis and database request analysis are associated, so that more comprehensive data information is obtained, and the accuracy of data traceability is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of information security technology, and in particular relates to a data interaction path association reasoning method and system based on multi-source heterogeneous logs. Background Art

[0002] As a new factor of production in the digital economy, data creates value through mobility and sharing, but it also faces significant risks of leakage. Data security has become a crucial component of national security. my country has successively issued a number of laws and regulations to strengthen data security supervision, of which data traceability is a crucial component. Traditional data traceability devices capture all input and output information flowing through HTTP (Hypertext Transfer Protocol) servers and database servers through methods such as traffic mirroring on the core switch side. Based on this information, they then determine the relationship between the HTTP server and the database table by comparing the ID (Identity) number and field content of the data records.

[0003] As data security levels continue to rise, traditional traffic analysis is no longer able to decrypt highly secure encrypted data traffic, nor can it directly forward data records containing private information to data traceability devices. On data traceability devices, HTTP server access logs and database access logs only record partial request and response information, excluding sensitive information. Consequently, HTTP request analysis and database request analysis can only operate independently, making it impossible to link HTTP request analysis and database request analysis together during data traceability to obtain more comprehensive data information, resulting in poor data traceability accuracy. Summary of the Invention

[0004] To overcome the above-mentioned deficiencies of the prior art, the present invention provides a data interaction path association reasoning method based on multi-source heterogeneous logs, comprising:

[0005] Generate an HTTP interaction log array list based on the log data of the HTTP server, construct a corresponding time bitmap according to the access time list in the HTTP interaction log array list, and generate an access path pattern array list according to the time bitmap and the HTTP interaction log array list;

[0006] Generate a database interaction log array list based on the database log data, construct a corresponding time bitmap according to the access time list in the database interaction log array list, and generate an access table pattern array list according to the time bitmap and the database interaction log array list;

[0007] Perform association reasoning on the access path pattern array list and the access table pattern array list to determine an association array list between the access path pattern list and the access table pattern list.

[0008] Preferably, the HTTP interaction log array list is generated based on the log data of the HTTP server, including:

[0009] Parse each log data of the HTTP server to obtain each parsed HTTP log data;

[0010] Normalize each HTTP log data after parsing to obtain each HTTP log array;

[0011] Obtain the HTTP interaction log array list according to each HTTP log array.

[0012] Preferably, each piece of HTTP log data after parsing includes timestamp, source IP, source port, destination IP, destination port, request URL, request length, response code, and response length;

[0013] Each HTTP log array includes access time, access action, access path, access requester ID, and access responder ID;

[0014] The normalization processing of each parsed HTTP log data includes:

[0015] The access time is generated based on the timestamp of the HTTP log data; the access action is generated based on the legal request method of the HTTP protocol of the HTTP log data; the access path is generated based on the URL in the HTTP request of the HTTP log data, and the access path includes the URL path and request parameters; the access requester identifier is generated based on the source IP of the HTTP request of the HTTP log data; and the access responder identifier is generated based on the destination IP and destination port of the HTTP log data.

[0016] Preferably, constructing a corresponding time bitmap according to the access time list in the HTTP interaction log array list, and generating an access path pattern array list according to the time bitmap and the HTTP interaction log array list comprises:

[0017] Modify the parameter value of the request parameter of the access path in the HTTP interaction log array list to the parameter name, and obtain the HTTP interaction log array list after parameter processing;

[0018] Aggregating the parameter-processed HTTP interaction log array list according to the access action and access path tuples, constructing a corresponding access time list, access requester identification list, and access responder identification list for each access action and access path tuple, and generating each access path pattern set;

[0019] Constructing a corresponding time bitmap based on the access time list in each access path pattern set, and generating a corresponding access path pattern array; wherein the time bitmap includes an access time bitmap and an extended access time bitmap;

[0020] The access path pattern array list is generated according to each access path pattern array.

[0021] Preferably, the database interaction log array list is generated based on the database log data, including:

[0022] Parse each log data in the database to obtain the parsed database log data;

[0023] Normalize each database log data after parsing to obtain each database log array;

[0024] The database interaction log array list is obtained according to each database log data array.

[0025] Preferably, the parsed log data of each database includes timestamp, source IP, source port, destination IP, destination port, and request information;

[0026] Each database log array includes access time, data processing action, access table identifier, access requester identifier, and access responder identifier;

[0027] The normalization processing of the parsed log data of each database includes:

[0028] The access time is generated based on the timestamp of the database log data; the statements in the log are converted into an abstract syntax tree in a statement syntax analyzer to extract the data processing action; the statements in the log are converted into an abstract syntax tree in a statement syntax analyzer to extract the access table identifier; the access requester identifier is generated based on the source IP of the database request of the database log data; and the access responder identifier is generated based on the destination IP and destination port of the database log data.

[0029] Preferably, constructing a corresponding time bitmap according to the access time list in the database interaction log array list, and generating an access table pattern array list according to the time bitmap and the database interaction log array list comprises:

[0030] Aggregate the database interaction log array list according to the data processing action and access table identifier tuple, construct a corresponding access time list, access requester identifier list, and access responder identifier list for each data processing action and access table identifier tuple, and generate each access table pattern set;

[0031] Constructing a corresponding time bitmap based on the access time list in each access table pattern set, and generating a corresponding access table pattern array; wherein the time bitmap includes an access time bitmap and an extended access time bitmap;

[0032] The access table pattern array list is generated according to each access table pattern array.

[0033] Preferably, performing association reasoning on the access path pattern array list and the access table pattern array list to determine an association array list between the access path pattern list and the access table pattern list includes:

[0034] Based on each of the access path pattern arrays, and according to the matching degree between the time bitmap in the access path pattern array and the time bitmap in each of the access table pattern arrays, a list of access table pattern arrays associated with the access path pattern array is screened out.

[0035] Preferably, the step of filtering out the access table pattern array list associated with the access path pattern array according to the matching degree between the time bitmap in the access path pattern array and the time bitmap in each access table pattern array comprises:

[0036] Performing a bitwise AND operation on the access time bitmap in the access path pattern array and the extended access time bitmap in each access table pattern array to obtain the number of 1s in each result bitmap;

[0037] Determine the ratio of the number of 1s in each result bitmap to the number of second values ​​in the access time bitmap in the access path pattern array as the matching degree between the access path pattern array and each access table pattern array;

[0038] The access table pattern arrays with matching degrees within a preset matching threshold range are determined as the access table pattern array list associated with the access path pattern array list.

[0039] Preferably, performing association reasoning on the access path pattern array list and the access table pattern array list to determine an association array list between the access path pattern list and the access table pattern list includes:

[0040] Based on each of the access table pattern arrays, an access path pattern array list associated with the access table pattern array is screened out according to a matching degree between a time bitmap in the access table pattern array and a time bitmap in each of the access path pattern arrays.

[0041] Preferably, the step of filtering out the access path pattern array list associated with the access table pattern array according to the matching degree between the time bitmap in the access table pattern array and the time bitmap in each access path pattern array comprises:

[0042] Performing a bitwise AND operation on the access time bitmap in the access table pattern array and the extended access time bitmap in each access path pattern array to obtain the number of 1s in each result bitmap;

[0043] Determine the ratio of the number of 1s in each result bitmap to the number of second values ​​in the access time bitmap in the access table pattern array as the matching degree between the access table pattern array and each access path pattern array;

[0044] The access path pattern arrays whose matching degrees are within a preset matching threshold range are determined as the access path pattern array list associated with the access table pattern array list.

[0045] Preferably, constructing a corresponding access time bitmap based on the access time list in each access path pattern set includes:

[0046] Selecting a time period including the access time in each of the access path pattern sets and the access time in each of the access table pattern sets, and numbering the time periods consecutively starting from 1 with one-second intervals to obtain a time period number;

[0047] Determine the size of the memory space according to the maximum number in the time period numbers, and set all initial values ​​in the memory space to the first value;

[0048] Traverse the access time in each of the access path pattern sets, convert the access time of each of the access path pattern sets into a time period number, set the time period number corresponding to the access time to a second value in the corresponding number position in the memory space, and obtain the access time bitmap corresponding to each of the access path pattern sets.

[0049] Preferably, constructing a corresponding extended access time bitmap based on the access time list in each access path pattern set includes:

[0050] Predetermine the number of seconds to delay the associated database access event;

[0051] Setting the number of rated digits following each second value of the access time bitmap to the second value, wherein the rated digits is the number corresponding to the number of seconds allowed to delay the associated database access event;

[0052] An extended access time bitmap of the access path pattern array is determined according to the access time bitmap after being set to a second value.

[0053] Preferably, constructing a corresponding access time bitmap based on the access time list in each access table pattern set includes:

[0054] Selecting a time period including the access time in each of the access table pattern sets and the access time in each of the access path pattern sets, and numbering the time periods consecutively starting from 1 with one second as an interval to obtain a time period number;

[0055] Determine the size of the memory space according to the maximum number in the time period numbers, and set all initial values ​​in the memory space to the first value;

[0056] Traverse the access time in each of the access table pattern sets, convert the access time of each of the access table path pattern sets into a time period number, set the time period number corresponding to the access time to a second value in the corresponding number position in the memory space, and obtain an access time bitmap corresponding to each of the access table pattern sets.

[0057] Preferably, constructing a corresponding extended access time bitmap based on the access time list in each access table pattern set includes:

[0058] Predetermine the number of seconds allowed for the associated HTTP request event to advance;

[0059] Setting the number of first rated digits of each second value in the access time bitmap to the second value, wherein the rated digits is the number corresponding to the number of seconds allowed for the associated HTTP request event to advance;

[0060] An extended access time bitmap of the access table pattern array is determined according to the access time bitmap after being set to a second value.

[0061] Based on the same inventive concept, the present invention also provides a data interaction path association reasoning system based on multi-source heterogeneous logs, the system comprising: an HTTP processing unit, a database processing unit, and an association reasoning unit; the HTTP processing unit is connected to the association reasoning unit, and the database processing unit is connected to the association reasoning unit;

[0062] An HTTP processing unit is configured to parse log data of an HTTP server, generate an HTTP interaction log array list, construct a corresponding time bitmap based on an access time list in the HTTP interaction log array list, generate an access path pattern array list based on the time bitmap and the HTTP interaction log array list, and send the access path pattern array list to an association reasoning unit;

[0063] a database processing unit, configured to parse database log data, generate a database interaction log array list, construct a corresponding time bitmap based on an access time list in the database interaction log array list, generate an access table pattern array list based on the time bitmap and the database interaction log list, and send the access table pattern array list to an association reasoning unit;

[0064] An associative reasoning unit is used to receive the access path pattern array list and the access table pattern array list, perform associative reasoning on the access path pattern array list and the access table pattern array list, and determine an associative array list between the access path pattern array list and the access table pattern array list.

[0065] Preferably, the HTTP processing unit is used to:

[0066] Parse each log data of the HTTP server to obtain each parsed HTTP log data;

[0067] Normalize each HTTP log data after parsing to obtain each HTTP log array;

[0068] Obtain the HTTP interaction log array list according to each HTTP log array.

[0069] Preferably, the database processing unit is used to:

[0070] Parse each log data in the database to obtain the parsed database log data;

[0071] Normalize each database log data after parsing to obtain each database log array;

[0072] Obtain the database interaction log array list according to each database log array.

[0073] Based on the same inventive concept, the present invention further provides an electronic device, comprising: at least one processor and a memory; the memory and the processor are connected via a bus;

[0074] The memory is used to store one or more programs;

[0075] When the one or more programs are executed by the at least one processor, the aforementioned data interaction path association reasoning method based on multi-source heterogeneous logs is implemented.

[0076] Based on the same inventive concept, the present invention also provides a readable storage medium having an execution program stored thereon. When the execution program is executed, the aforementioned data interaction path association reasoning method based on multi-source heterogeneous logs is implemented.

[0077] Compared with the closest prior art, the present invention has the following beneficial effects:

[0078] The present invention provides a data interaction path association reasoning method and system based on multi-source heterogeneous logs. The method comprises: generating an HTTP interaction log array list based on HTTP server log data, constructing a corresponding time bitmap according to an access time list in the HTTP interaction log array list, and generating an access path pattern array list according to the time bitmap and the HTTP interaction log array list; generating a database interaction log array list based on database log data, constructing a corresponding time bitmap according to an access time list in the database interaction log array list, and generating an access table pattern array list according to the time bitmap and the database interaction log array list; performing association reasoning on the access path pattern array list and the access table pattern array list, and determining an association array list between the access path pattern list and the access table pattern list. The present invention associates HTTP request analysis and database request analysis by time to obtain the data flow path on the HTTP server and the database server, thereby obtaining more comprehensive data information, improving the accuracy of data traceability, and further strengthening the ability of data security supervision. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 The present invention provides a flow chart of a data interaction path association reasoning method based on multi-source heterogeneous logs;

[0080] Figure 2 This is a structural diagram of a data interaction path association reasoning system based on multi-source heterogeneous logs provided by the present invention;

[0081] Figure 3 A schematic diagram of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0082] Traditional data tracing devices capture all input and output information flowing through the HTTP server and database server through means such as traffic mirroring on the core switch side, and then determine the relationship between the HTTP server path and the database table by comparing the ID number and field content of the data records.

[0083] However, with the continuous improvement of data security levels, traditional traffic analysis can no longer decrypt encrypted data traffic with high security levels (such as the forward-secure session key negotiation protocol enabled by default in TLS (Transport Layer Security) 1.3 and above, which cannot be decrypted by importing public and private keys afterwards). The requirements for data privacy also prevent users from directly forwarding data records containing privacy information to the data traceability device without filtering.

[0084] Traditional content-based association algorithms for HTTP and database requests consist of two steps: content feature extraction and similar feature merging. Content feature extraction extracts the substantive content from HTTP request-response logs and database request-response logs, and then correlates HTTP request-response logs and database request-response logs with identical substantive content. This suggests that traditional content-based association algorithms require more log data information. Current HTTP server access logs and database access logs only record partial request and response information, rarely including sensitive information. Therefore, the lack of log data renders traditional content-based association algorithms for HTTP and database requests ineffective. With only log data, the data traceability device's HTTP request analysis and database request analysis can only operate independently, unable to further explore the associated data flow relationships between the HTTP and database servers. Consequently, when performing data traceability, HTTP request analysis and database request analysis cannot be correlated to obtain more comprehensive data information, resulting in poor data traceability accuracy.

[0085] Based on this, in an embodiment of the present invention, by associating HTTP request analysis and database request analysis through time, the flow path of data on the HTTP server and the database server is obtained, thereby obtaining more comprehensive data information, improving the accuracy of data traceability, and further strengthening the ability of data security supervision.

[0086] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0087] Example 1:

[0088] The present invention provides a data interaction path association reasoning method based on multi-source heterogeneous logs, specifically, Figure 1A flowchart of a data interaction path association reasoning method based on multi-source heterogeneous logs provided by an embodiment of the present invention includes the following steps as shown in the figure:

[0089] Step S1: Generate an HTTP interaction log array list based on the log data of the HTTP server, construct a corresponding time bitmap according to the access time list in the HTTP interaction log array list, and generate an access path pattern array list according to the time bitmap and the HTTP interaction log array list;

[0090] Step S2: Generate a database interaction log array list based on the database log data, construct a corresponding time bitmap according to the access time list in the database interaction log array list, and generate an access table pattern array list according to the time bitmap and the database interaction log array list;

[0091] Step S3: performing association reasoning on the access path pattern array list and the access table pattern array list to determine an association array list between the access path pattern list and the access table pattern list.

[0092] The present invention associates HTTP request analysis and database request analysis through time to obtain the flow path of data on the HTTP server and the database server, thereby obtaining more comprehensive data information, improving the accuracy of data tracing, and further strengthening the ability of data security supervision.

[0093] In the present invention, step S1-1 generates an HTTP interaction log array list based on the log data of the HTTP server. Specifically:

[0094] S1-1-a, parsing each log data of the HTTP server to obtain each parsed HTTP log data;

[0095] S1-1-b, normalize each HTTP log data after parsing to obtain an array of each HTTP log;

[0096] S1-1-c, obtain the HTTP interaction log array list according to each of the above HTTP log arrays.

[0097] In S1-1-a, each log data of the HTTP server is parsed, and each log data of the HTTP servers from multiple different sources can be parsed, wherein the log data from multiple different sources can be the log data of the HTTP servers in the microservice cluster composed of different HTTP servers. For example, service A uses Tomcat, SpringBoot and Log4j for logging, and service B uses the web framework of Nginx and node.js for logging. In other words, the log data of the HTTP server is not limited to a certain type of HTTP server log data, and any feasible HTTP server log data is within the scope of protection requested by this application. After parsing each log data of the HTTP servers from multiple different sources, each parsed HTTP log data can be obtained. Each parsed log data includes timestamp, source IP (Internet Protocol), source port, destination IP, destination port, request URL (Uniform Resource Locator), request length, response code, response length, etc.

[0098] Parsing HTTP server logs can use corresponding regular expressions to extract the required parts from log files of different sources, and filter out invalid requests whose response codes are not "200". Among them, regular expressions are a tool for matching and processing text, which can define rules and patterns to find, replace and extract target data. The HTTP status code "200" represents a valid request, which is successfully processed by the server; HTTP status codes other than "200" represent invalid requests, which are filtered out. For example, the HTTP status code 400 means that the requested URL cannot be found on the server; the HTTP status code 403 means that the server denies access or does not have sufficient access rights; the HTTP status code 500 represents an internal server error.

[0099] The process of normalizing each parsed log data in S1-1-b may include: generating the access time based on the timestamp of the HTTP log data; generating the access action based on the legal request method of the HTTP protocol of the HTTP log data; generating the access path based on the request URL of the HTTP log data, the access path including the URL path and request parameters; generating the access requester identifier based on the source IP of the HTTP request; generating the access responder identifier based on the destination IP and destination port of the HTTP log data.

[0100] The process of generating access times based on HTTP log data timestamps involves converting log timestamps into 64-bit unsigned UNIX timestamps for storage, removing the log file's offset from standard time, and obtaining the access time. If there are time zone differences, the access time is converted to UTC+0. The UNIX timestamp is the number of seconds since January 1, 1970 (midnight UTC / GMT), excluding leap seconds.

[0101] Generate access actions based on the legitimate request methods of the HTTP protocol based on HTTP log data. The legitimate request methods of the HTTP protocol can be GET / POST / HEAD / PUT / DELETE / OPTIONS / TRACE, etc. Among them, the GET request method is used to request the specified page information and return the entity body. The HEAD request method is used to request the specified page information. There is no specific content in the returned response. It is used to obtain the header. The POST request method is used to submit data to the specified resource for processing the request (such as submitting a form or uploading a file). The PUT request method is used to transmit data from the client to the server to replace the content of the specified document. The DELETE request method is used to request the server to delete the specified page. The OPTIONS request method is used to allow the client to view the performance of the server. The TRACE request method is used to echo the request received by the server.

[0102] The access path is generated based on the URL in the HTTP request in HTTP log data. The access path consists of the URL path (Path) and request parameters (Param). The URL path (Param) can be in the following forms: no slash, single slash ( / ), dot + single slash (. / ), dot + single slash (.. / ), multiple dots + single slash (.. / .. / ), and the full path. No slash and dot + single slash (. / ) indicate a jump to the layout page in the same directory as the root directory; a single slash ( / ) jumps to the root directory of the entire website; dot + single slash (.. / ) jumps to the parent directory; multiple dots + single slash (.. / .. / ) jumps up one level with each .. / , and the number of .. / s increases the number of levels. The full path is the path + project name + the file's location in the webapp. Request parameters (Param) are data passed as part of the URL path, typically used to identify the uniqueness of resources or hierarchical relationships. For example, for the URL http: / / example.com / users / 123, 123 is a Params parameter used to identify the user with user ID 123.

[0103] Generates an access requester ID based on the source IP address of the HTTP request in the HTTP log data. The source IP address can uniquely identify a host on the network, indicating the machine from which the information originated.

[0104] Generate an access responder ID based on the destination IP and destination port of the HTTP log data. The destination IP indicates the process to which the information is destined. The destination port number indicates the machine to which the information is destined. If the access responder ID cannot be extracted from a single record in the log file, the user can submit the IP address and port number of the HTTP server where the HTTP server log file is located as the access responder ID for each log entry in the log file.

[0105] S1-1-b normalizes each parsed HTTP log data entry. Each resulting log array includes the access time, access action, access path, access requester ID, and access responder ID. Specifically, each log array forms a five-tuple consisting of (access time, access action, access path, access requester ID, and access responder ID). It will be appreciated that this five-tuple is merely an example, which will be used in subsequent explanations and is not intended to limit this application.

[0106] In addition to access time, access action, access path, access requester ID, and access responder ID, the normalized log array may also include other normalized log data and may also form other multi-element arrays, without specific limitation.

[0107] S1-1-c, obtain an HTTP interaction log array list based on each HTTP log array.

[0108] Obtain a multi-element array list according to multiple log arrays, and use the obtained multi-element array list as an HTTP interaction log array list.

[0109] Exemplarily, each HTTP log array can form a five-tuple of (access time, access action, access path, access requester ID, access responder ID), and multiple five-tuples are combined to form a five-tuple list, which is then used as the HTTP interaction log array list. The access times in the multiple five-tuples constitute the access time list in the HTTP interaction log array list, the access requester IDs in the multiple five-tuples constitute the access requester ID list in the HTTP interaction log array list, and the access responder IDs in the multiple five-tuples constitute the access responder ID list in the HTTP interaction log array list.

[0110] Table 1 shows an example of an HTTP interaction log array list.

[0111] Access time Access Action Access Path Access requester ID Access responder ID Quintuple 1 T1 A1 R1 D1 E1 Quintuple 2 T2 A2 R2 D2 E2 Quintuple 3 T3 A3 R3 D3 E3 Quintuple 4 T4 A4 R4 D4 E4 Quintuple 5 T5 A5 R5 D5 E5 …… …… …… …… …… …… Quintuple n Tn An Rn Dn En

[0112] Table 1

[0113] As shown in Table 1, horizontally, (T1, A1, R1, D1, E1) constitutes a quintuple, and (Tn, An, Rn, Dn, En) constitutes another quintuple; vertically, (T1, T2, T3, …, Tn) constitutes the access time list, (D1, D2, D3, …, Dn) constitutes the access requester identification list, and (E1, E2, E3, …, En) constitutes the access responder identification list.

[0114] After step S1-1 generates an HTTP interaction log array list based on the log data of the HTTP server, step S1-2 constructs a corresponding time bitmap according to the access time list in the HTTP interaction log array list, and generates an access path pattern array list based on the time bitmap and the HTTP interaction log array list.

[0115] S1-2-a, modify the parameter value of the request parameter of the access path in the HTTP interaction log array list to the parameter name, and obtain the HTTP interaction log array list after parameter processing.

[0116] Modify the variable parameter value portion of the request parameter of the access path in the HTTP interaction log array list to a formal parameter name. Specifically, first predetermine a set of two-tuple lists, each of which includes two elements: a formal parameter name and a parameter regular expression. Then, match the parameter regular expression in the two-tuple list one by one for the access path, and replace the substring that matches the parameter regular expression with the formal parameter name corresponding to the parameter regular expression. For example, common parameter names such as ID, EMAIL, and UUID (Universally Unique Identifier) ​​have corresponding regular expressions such as " / \d+", " / \w+@[a-zA-Z_]+?\.[a-zA-Z]{2,3}$", and " / ((?-i:0x)?[A-Fa-f0-9]{32}|"

[0117] [A-Fa-f0-9]{8}-[A-Fa-f0-9]{4}-[A-Fa-f0-9]{4}-[A-Fa-f0-9]{4}-[A-Fa-f0-9]{12}|

[0118] \{[A-Fa-f0-9]{8}-[A-Fa-f0-9]{4}-[A-Fa-f0-9]{4}-[A-Fa-f0-9]{12}\})$", etc. At this time, the access path " / addTask / 12021 / ? attachment=3424837E-8C51-D183-B432-11D05CF126DE" can be processed into " / addTask / ID / ? attachment=UUID".

[0119] For example, the original API (Application Program Interface) record presented in the log may be in the form of ( / download / aaa.txt, / download / bbb.pdf, / download / ccc.jpg), where the variable parameter value part in the request parameter is aaa.txt, bbb.pdf, ccc.jpg, and the above variable parameter value parts are replaced with the formal parameter name using the parameter regular expression. <filename>Finally, all three APIs are replaced with / download / <filename>In this way, these three APIs will be regarded as the same API ( / download / <filename>), this API will be recorded as appearing three times when associated with the access table pattern array in subsequent steps. Without replacing the variable parameter values ​​with formal parameter names using a regular expression, these three APIs would be treated as separate APIs and analyzed. Each API would appear only once, making it impossible to accumulate the number of occurrences when associated with the access table pattern array. By replacing the parameter values ​​of the access path request parameters in the HTTP interaction log array with parameter names, the complexity of subsequent operations is reduced.

[0120] S1-2-b, aggregate the HTTP interaction log array list after parameter processing according to the access action and access path tuples, construct the corresponding access time list, access requester identification list, and access responder identification list for each access action and access path tuple, and generate each access path pattern set.

[0121] Table 2 shows an example of an array list of HTTP interaction logs after parameter processing.

[0122] Access time Access Action Access Path Access requester ID Access responder ID Quintuple 1 T1 a1 r1 D1 E1 Quintuple 2 T2 a1 r1 D2 E2 Quintuple 3 T3 a2 r2 D3 E3 Quintuple 4 T4 a1 r1 D4 E4 Quintuple 5 T5 a2 r2 D5 E5 …… …… …… …… …… …… Quintuple n Tn An Rn Dn En

[0123] As can be seen from Table 2, the access actions and access paths of quintuples 1, 2, and 4 in the HTTP interaction log array list after parameter processing are considered to be the same, and the access actions and access paths of quintuples 3 and 5 are considered to be the same. According to the access action and access path tuples, the obtained quintuples 1, 2, and 4 are (a1, r1), and the corresponding access time list is You can also use the transposed matrix (T1, T2, T4) T To indicate that the access requester identification list is You can also use the transposed matrix (D1, D2, D4) T To indicate that the access responder identification list is You can also use the transposed matrix (E1, E2, E4) T Similarly, the binary tuple of 5 and 3 is (a2, r2), and the corresponding access time list is (T3, T5). T , the access requester identification list is (D3, D5) T , the access responder identification list is (E3, E5) T .

[0124] That is, the access path pattern set consisting of the quintuples 1, 2, and 4 is:

[0125] The access path pattern set consisting of quintuples 3 and 5 is:

[0126] S1-2-b is used to aggregate tuples with the same or considered to be the same access actions and access paths, obtain the access time list, access requester identification list, and access responder identification list constructed for the same access action and access path, and generate each access path pattern set based on this.

[0127] S1-2-c is used to construct a corresponding time bitmap based on the access time list in each access path pattern set and generate a corresponding access path pattern array; the time bitmap includes an access time bitmap and an extended access time bitmap.

[0128] The corresponding access time bitmap is constructed based on the access time list in each access path pattern set, including:

[0129] Select a time period that includes the access time in each access path pattern set and the access time in each access table pattern set, and number the time periods consecutively starting from 1 with one-second intervals to obtain a time period number;

[0130] Determine the size of the memory space according to the maximum number in the time period number, and set all initial values ​​in the memory space to the first value;

[0131] Traverse the access time in each access path pattern set, convert the access time of each access path pattern set into a time period number, set the time period number corresponding to the access time to the second value in the corresponding number position in the memory space, and obtain the access time bitmap corresponding to each access path pattern set.

[0132] Specifically, a time period is first selected that encompasses all access times in the access path pattern set and all access times in the access table pattern set. Each second in this time period is consecutively numbered, starting from 1. Assuming the time period is N seconds long, the time periods are numbered 1, 2, 3, ..., N. The maximum time period is N, and the starting access time is S (the values ​​of N and S remain consistent throughout a complete multi-source heterogeneous log data interaction path association inference). The user then preselects a time period that encompasses as many access times as possible in the access path pattern set and all access table pattern sets. This selected time period is not completely precise, so a longer time period is selected to achieve a larger time range, such as one month. Next, a memory space M is allocated (the starting bit of M is the first bit), its bit size is set to N, and all bits are initially set to 0. This data structure can be efficiently implemented using the Bitset class in the C++ standard library. Finally, all access times in each access path pattern set are traversed, and each access time, which is accurate to the second, is converted into a time period second number. The corresponding number position in the memory space is set to 1. That is, for each access time X, the value of the bit at position X-S+1 in M ​​is set to 1. The memory space obtained at this point is the access time bitmap constructed based on the access time list in each access path pattern set.

[0133] Furthermore, constructing a corresponding extended access time bitmap based on the access time list in each access path pattern set includes:

[0134] Predetermine the number of seconds to delay the associated database access event;

[0135] The number of rated digits following each second value of the access time bitmap is set to the second value, where the rated digits are the number of seconds corresponding to the delay allowed for the associated database access event;

[0136] An extended access time bitmap of the access path pattern array is determined according to the access time bitmap after being set to the second value.

[0137] Specifically, the number of seconds D allowed for the associated database access event to lag is predetermined (the value of D remains consistent throughout a complete multi-source heterogeneous log data interaction path association inference); the next several bits following each 1 bit in the access time bitmap are set to 1, that is, for each access time X, the bits from X-S+1 to X-S+D+1 are set to 1, corresponding to the number of seconds of lag in the associated database access event, thereby constructing a corresponding extended access time bitmap.

[0138] In another specific implementation, a corresponding extended access time bitmap may be constructed based on the access time list in each access path pattern set.

[0139] Specifically, the number of seconds D allowed to lag associated database access events is predetermined (the value of D remains consistent throughout a complete multi-source heterogeneous log data interaction path association inference); then, a memory space M' is allocated (the starting bit of M' is the first bit), the bit size of the memory space is set to N, and the initial values ​​of all bits in the space are set to 0. This data structure can be efficiently implemented using the Bitset class in the C++ standard library; all access times in each access path pattern set are traversed, and for each access time X, bits X-S+1 to X-S+D+1 in M' are set to 1 (if any value from X-S+2 to X-S+D+1 is greater than X+N+1, the value is not set to 1), and the corresponding extended access time bitmap is constructed.

[0140] S1-2-c is used to construct a corresponding access time bitmap and an extended access time bitmap for the access time list in each generated access path pattern set, and generate a corresponding access path pattern array.

[0141] For example, for each (access action, access path) tuple, there is a corresponding (access time bitmap, extended access time bitmap, access requester identification list, access responder identification list) tuple. For example, in the access path pattern set composed of the above five tuples 1, 2, and 4, the generated access path pattern array is You can also Where Tm1 represents the access time bitmap corresponding to the access path pattern array, and Tn1 represents the extended access time bitmap corresponding to the access path pattern array.

[0142] S1-2-d, generating an access path pattern array list according to each access path pattern array.

[0143] Traverse each (access action, access path) two-tuple, and merge the obtained (access time bitmap, extended access time bitmap, access requester identification list, access responder identification list) four-tuple into (access action, access path, access time bitmap, extended access time bitmap, access requester identification list, access responder identification list) six-tuple to obtain the access path pattern array list. It can be understood that the access path pattern array list contains multiple access path pattern arrays, for example, it can include Can also include Wherein, Tm1 represents the access time bitmap corresponding to the access path pattern array, and Tn1 represents the extended access time bitmap corresponding to the access path pattern array; Tm2 represents the access time bitmap corresponding to the access path pattern array, and Tn2 represents the extended access time bitmap corresponding to the access path pattern array. Step S1 first generates an HTTP interaction log array list based on the log data of the HTTP server, and then constructs a corresponding time bitmap according to the access time list in the HTTP interaction log array list, and finally generates an access path pattern array list according to the time bitmap and the HTTP interaction log array list. Step S1 constructs the corresponding access time bitmap and the extended access time bitmap through the access time list in the HTTP interaction log array list, which greatly reduces the time complexity of the time matching search based on adjacent time, and at the same time provides data support for the subsequent automatic discovery of the association relationship between the data interaction path of the HTTP server and the database server through the time association relationship.

[0144] In the present invention, step S2-1 generates a database interaction log array list based on the database log data. Specifically:

[0145] S2-1-a, parsing each log data of the database to obtain each parsed database log data;

[0146] S2-1-b, normalizing each piece of database log data after parsing to obtain an array of each database log;

[0147] S2-1-c, obtaining the database interaction log array list according to each database log array.

[0148] In S2-1-a, each log data item of the database server is parsed, and each log data item of database servers from multiple different sources can be parsed, wherein the database log data from multiple different sources can be the log data of database servers in a microservice cluster composed of different database servers. For example, service A uses Tomcat, SpringBoot, and Log4j for logging, and service B uses the web framework of Nginx and node.js for logging. In other words, the log data of the database server is not limited to a certain type of database server log data, and any feasible database server log data is within the scope of protection requested in this application.

[0149] S2-1-a parses each log data entry from various database servers to obtain parsed database log data. Database log parsing can extract the required components from log files of different sources using corresponding regular expressions. Each parsed database log data entry includes timestamp, source IP address, source port, destination IP address, destination port, and request information.

[0150] The process of S2-1-b for normalizing each parsed log data may include: generating the access time based on the timestamp of the database log data; converting the statements in the log into an abstract syntax tree in the statement syntax analyzer to extract the data processing action; converting the statements in the log into an abstract syntax tree in the statement syntax analyzer to extract the access table identifier; generating the access requester identifier based on the source IP of the database request of the database log data; generating the access responder identifier based on the destination IP and destination port of the database log data.

[0151] The process of generating the access time based on the timestamp of the database log data can be to convert the log timestamp into a 64-bit unsigned integer for storage, and then eliminate the offset of the log file relative to the standard time to obtain the access time. If there is a time zone difference, it is unified to UTC+0 standard time. The UNIX timestamp is the number of seconds that have passed since January 1, 1970 (midnight UTC / GMT), not taking leap seconds into account.

[0152] For data processing actions and access table identifiers, the statement in the log can be converted into an abstract syntax tree in a statement parser to extract the data processing action. The SQL (Structured Query Language) statement parser can be used to convert the SQL statement in the log into an abstract syntax tree (AST), from which the data processing action (Data Manipulation Statement, i.e., Select / Insert / Update / Delete) and access table identifier (TableIdentifier) ​​are extracted, and logs with non-data processing statements (such as Set / Connect and other database maintenance statements) are ignored. Among them, the AST is an abstract representation of the grammatical structure of the source code. It can be understood that if there are multiple access table identifiers in a database log data, the database log data is split into multiple records, each record corresponding to an access table identifier.

[0153] Generate an access requester ID based on the source IP address of the database request in the database log data. The source IP address can uniquely identify a host in the network, indicating which machine the information originated from.

[0154] Generate an access responder ID based on the destination IP and destination port of the database log data. The destination IP indicates the process to which the information is destined. The destination port indicates the machine to which the information is destined. If the access responder ID cannot be extracted from a single record in the log file, the user can submit the IP address and port number of the database server where the database server log file is located as the access responder ID for each log entry in the log file.

[0155] S2-1-b normalizes each parsed database log data entry. The resulting database log array includes the access time, data processing action, access table identifier, access requester identifier, and access responder identifier. Specifically, each log array forms a five-tuple consisting of (access time, data processing action, access table identifier, access requester identifier, and access responder identifier). It will be appreciated that this five-tuple is merely an example and will be used in subsequent explanations. It is not intended to limit this application.

[0156] In addition to access time, data processing action, access table identifier, access requester identifier, and access responder identifier, the normalized log array may also include other normalized log data and may also form other multi-element arrays, without specific limitation.

[0157] S2-1-c, obtaining a database interaction log array list according to each database log array.

[0158] A multi-element array list is obtained according to the multiple log arrays, and the obtained multi-element array list is used as a database interaction log array list.

[0159] Exemplarily, each database log array can form a five-tuple of (access time, data processing action, access table identifier, access requester identifier, access responder identifier), and multiple five-tuples are combined to form a five-tuple list, which is then used as the database interaction log array list. The access times in the multiple five-tuples constitute the access time list in the database interaction log array list, the access requester identifiers in the multiple five-tuples constitute the access requester identifier list in the database interaction log array list, and the access responder identifiers in the multiple five-tuples constitute the access responder identifier list in the database interaction log array list.

[0160] Table 3 shows an example of a database interaction log array list.

[0161]

[0162]

[0163] Table 3

[0164] As shown in Table 3, horizontally, (T′1, A′1, R′1, D′1, E′1) constitutes a quintuple, and (T′1, T′2, T′3, …, T′n) constitutes the access time list; vertically, (D′1, D′2, D′3, …, D′n) constitutes the access requester identification list, and (E′1, E′2, E′3, …, E′n) constitutes the access responder identification list.

[0165] After step S2-1 generates a database interaction log array list based on the log data of the database server, step S2-2 constructs a corresponding time bitmap according to the access time list in the database interaction log array list, and generates an access table pattern array list based on the time bitmap and the database interaction log array list.

[0166] S2-2-a, aggregate the database interaction log array list according to the data processing action and access table identifier tuple, construct the corresponding access time list, access requester identifier list, and access responder identifier list for each data processing action and access table identifier tuple, and generate each access table pattern set.

[0167] As can be seen from Table 3, the data processing actions and access table identifiers of quintuples 1, 2, and 4 in the database interaction log data list are the same, and the data processing actions and access table identifiers of quintuples 3 and 5 are the same. According to the data processing action and access table identifier binary tuple aggregation operation, the binary tuple of quintuples 1, 2, and 4 is obtained as (A′1, R′1), and the corresponding access time list is You can also use the transposed matrix (T'1, T'2, T'4) T To indicate that the access requester identification list is You can also use the transposed matrix (D′1, D′2, D′4) T To indicate that the access responder identification list is You can also use the transposed matrix (E'1, E'2, E'4) T Similarly, the binary tuple of 5-tuple 3 and 5 is (A′2, R′2), and the corresponding access time list is (T′3, T′5). T , the access requester identification list is (D′3, D′5) T , the access responder identification list is (E′3, E′5) T .

[0168] That is, the access table pattern set consisting of quintuples 1, 2, and 4 is

[0169] The access table pattern set consisting of quintuples 3 and 5 is:

[0170] S2-2-a is used to aggregate tuples with the same data processing action and access table identifier to obtain an access time list, an access requester identifier list, and an access responder identifier list constructed for the same data processing action and access table identifier, and generate each access table pattern set accordingly.

[0171] S2-2-b is used to construct a corresponding time bitmap based on the access time list in each access table pattern set, and generate a corresponding access table pattern array; the time bitmap includes an access time bitmap and an extended access time bitmap.

[0172] The step of constructing a corresponding access time bitmap based on the access time list in each access table pattern set includes:

[0173] Select a time period that includes the access time in each access table pattern set and the access time in each access path pattern set, and number the time periods consecutively starting from 1 with one-second intervals to obtain a time period number;

[0174] Determine the size of the memory space according to the maximum number in the time period number, and set all initial values ​​in the memory space to the first value;

[0175] Traverse the access time in each access table pattern set, convert the access time of each access table path pattern set into a time period number, set the time period number corresponding to the access time to the second value in the corresponding number position in the memory space, and obtain the access time bitmap corresponding to each access table pattern set.

[0176] Specifically, a time period is first selected that encompasses all access times in the access path pattern set and all access times in the access table pattern set. Each second in this time period is consecutively numbered, starting from 1. Assuming the time period is N seconds long, the time periods are numbered 1, 2, 3, ..., N. The maximum time period is N, and the starting access time is S (the values ​​of N and S remain consistent throughout a complete multi-source heterogeneous log data interaction path association inference). The user then preselects a time period that encompasses as many access times as possible in the access path pattern set and all access table pattern sets. This selected time period is not completely precise, so a longer time period is selected to achieve a larger time range, such as one month. Next, a memory space M is allocated (the starting bit of M is the first bit), its bit size is set to N, and all bits are initially set to 0. This data structure can be efficiently implemented using the Bitset class in the C++ standard library. Finally, all access times in each access table pattern set are traversed, and each access time accurate to the second is converted into a time period second number. The corresponding number position in the memory space is set to 1. That is, for each access time X, the value of the bit at position X-S+1 in M ​​is set to 1. The memory space obtained at this time is the access time bitmap constructed based on the access time list in each access table pattern set.

[0177] Furthermore, constructing a corresponding extended access time bitmap based on the access time list in each access table pattern set includes:

[0178] Predetermine the number of seconds allowed for the associated HTTP request event to advance;

[0179] The first rated number of digits of each second value of the access time bitmap is set to the second value, where the rated number of digits corresponds to the number of seconds allowed for the associated HTTP request event to advance;

[0180] An extended access time bitmap of the access table pattern array is determined according to the access time bitmap after being set to the second value.

[0181] Specifically, the number of seconds D that the associated HTTP request event is allowed to advance is determined in advance (the value of D remains consistent in a complete multi-source heterogeneous log data interaction path association inference); the first several bits of each 1 bit in the access time bitmap are set to 1, that is, for each access time X, the bits from XS-D+1 to X-S+1 are set to 1 to correspond to the number of seconds that the associated HTTP request event is advanced, and the corresponding extended access time bitmap is constructed.

[0182] In another specific implementation, a corresponding extended access time bitmap may be constructed based on the access time list in each access table pattern set.

[0183] Specifically, the number of seconds D allowed for the associated HTTP request event to advance is predetermined (the value of D remains consistent throughout a complete multi-source heterogeneous log data interaction path association inference); then, a memory space M' is allocated (the starting bit of M' is the first bit), the bit size of the memory space is set to N, and the initial values ​​of all bits in the space are set to 0. This data structure can be efficiently implemented using the Bitset class in the C++ standard library; all access times in each of the access table pattern sets are traversed, and for each access time X, bits from XS-D+1 to X-S+1 in M' are set to 1 (if any value from XS-D+1 to X-S+1 is less than 1, the value is not set to 1), and the corresponding extended access time bitmap is constructed.

[0184] S2-2-b is used to construct a corresponding access time bitmap and an extended access time bitmap for the access time list in each generated access table pattern set, and generate a corresponding access table pattern array.

[0185] For example, for each (data processing action, access table identifier) ​​tuple, there is a corresponding (access time bitmap, extended access time bitmap, access requester identifier list, access responder identifier list) tuple. For example, in the access table pattern set composed of the above five tuples 1, 2, and 4, the generated access table pattern array is You can also Wherein Tm′1 represents the access time bitmap corresponding to the access table pattern array, and Tn′1 represents the extended access time bitmap corresponding to the access table pattern array.

[0186] S2-2-c, generating an access path pattern array list according to each access path pattern array.

[0187] Traverse each (data processing action, access table identifier) ​​two-tuple, and merge the obtained (access time bitmap, extended access time bitmap, access requester identifier list, access responder identifier list) four-tuple into (data processing action, access table identifier, access time bitmap, extended access time bitmap, access requester identifier list, access responder identifier list) six-tuple to obtain the access table pattern array list. It can be understood that the access table pattern array list contains multiple access table pattern arrays, for example, it can include Can also include Wherein, Tm′2 represents the access time bitmap corresponding to another access table pattern array, and Tn′2 represents the extended access time bitmap corresponding to another access table pattern array.

[0188] Step S2 first generates a database interaction log array list based on the database server's log data, then constructs a corresponding time bitmap based on the access time list in the database interaction log array list, and finally generates an access table pattern array list based on the time bitmap and the database interaction log array list. Step S2 constructs the corresponding access time bitmap and extended access time bitmap based on the access time list in the database interaction log array list. This significantly reduces the time complexity of time matching searches based on adjacent time, and also provides data support for the subsequent automated discovery of the association between data interaction paths between the database server and the HTTP server through time association.

[0189] Step S3: performing association reasoning on the access path pattern array list and the access table pattern array list to determine an association array list between the access path pattern list and the access table pattern list.

[0190] The access path pattern array list and the access table pattern array list are cross-compared to obtain an association array list between the access path pattern list and the access table pattern list. Specifically, association reasoning is performed on the access path pattern array list and the access table pattern array list to determine that the association array lists between the access path pattern list and the access table pattern list include two types:

[0191] The first type is based on each access path pattern array and selects a list of access table pattern arrays associated with the access path pattern array based on the degree of match between the time bitmap in the access path pattern array and the time bitmap in each access table pattern array. Specifically, based on each access path pattern array, all access table pattern array lists are traversed, and several associated access table pattern arrays that match each access path pattern array are selected to obtain a list of access table pattern arrays associated with the access path pattern array.

[0192] The second type is based on each access table pattern array and, based on the degree of matching between the time bitmap in the access table pattern array and the time bitmap in each access path pattern array, filters out a list of access path pattern arrays associated with the access table pattern array. Specifically, using each access table pattern array as a benchmark, all access path pattern array lists are traversed, and a number of associated access path pattern arrays that match each access table pattern array are filtered to obtain a list of access path pattern arrays associated with the access table pattern array.

[0193] In the first type, based on the matching degree between the time bitmap in the access path pattern array and the time bitmap in each access table pattern array, a list of access table pattern arrays associated with the access path pattern array is screened, including:

[0194] Perform a bitwise AND operation on the access time bitmap in the access path pattern array and the extended access time bitmap in each access table pattern array to obtain the number of 1s in each resulting bitmap;

[0195] Determine the ratio of the number of 1s in each result bitmap to the number of second values ​​in the access time bitmap in the access path pattern array as the matching degree between the access path pattern array and each access table pattern array;

[0196] The access table pattern arrays whose matching degrees are within a preset matching threshold range are determined as the access table pattern array list associated with the access path pattern array list.

[0197] The preset matching threshold range can be a fixed ratio, for example, determining the access table pattern arrays with matching degrees in the top 20% as the access table pattern array list associated with the access path pattern array list; the preset matching threshold range can also be a fixed number, for example, determining the access table pattern arrays with matching degrees in the top 5 as the access table pattern array list associated with the access path pattern array list. The preset matching threshold range can be set by the user and is not limited here. Any feasible method for determining the preset matching threshold range is within the scope of protection requested by this application.

[0198] The bitwise AND operation involved in the above process means that the result of the bitwise AND operation is 1 if and only if the result of the access time bitmap in the access path pattern array and the result of the extended access time bitmap in the access table pattern array are both 1. That is, 1 & 1 == 1, 1 & 0 == 0, 0 & 1 == 0, and 0 & 0 == 1. In other words, each bit in the access time bitmap result in the access path pattern array is ANDed with each bit in the extended access time bitmap result in each access table pattern array to obtain the number of 1s in each result bitmap.

[0199] In a specific embodiment, for each access path pattern sextuple (access action, access path, access time bitmap, extended access time bitmap, access requester identification list, access responder identification list), several associated access table pattern sextuples (data processing action, access table identification, access time bitmap, extended access time bitmap, access requester identification list, access responder identification list) are filtered out according to the matching degree between the access time bitmap and the extended access time bitmap.

[0200] The specific steps are as follows:

[0201] (1) Count the number of 1s in the access time bitmap of the access path pattern 6-tuple, denoted as P;

[0202] (2) Perform a bitwise AND operation on the access time bitmap in the access path pattern sextuple and the extended access time bitmap of each access table pattern sextuple (assuming the number of access table pattern sextuples is L), and count the number of 1s in the resulting bitmap in turn, which are recorded as Q1, Q2, ..., Q L ;

[0203] (3) Q1, Q2, ..., Q L Divide by P respectively to get the matching degree V1, V2, ..., V of the 1st to Lth access table pattern six-tuples. L , where V1=Q1 / P、V2=Q2 / P、……V L =Q L / P;

[0204] (4) Select V1, V2, ..., V L The corresponding access table pattern six-tuples of the several largest median values ​​are recorded as the associated access table pattern six-tuples of the access path pattern six-tuple, forming an associated access table pattern six-tuple list.

[0205] The purpose of using access time bitmap and extended access time bitmap is to efficiently calculate the frequency of "access paths and access tables between HTTP log data and database log data appearing in the same time period". This eliminates the need for frequent log loops required by traditional calculations, thereby improving computing performance.

[0206] In the second type, a list of access path pattern arrays associated with the access table pattern array is screened based on the matching degree between the time bitmap in the access table pattern array and the time bitmap in each access path pattern array, including:

[0207] Perform a bitwise AND operation on the access time bitmap in the access table pattern array and the extended access time bitmap in each access path pattern array to obtain the number of 1s in each resulting bitmap;

[0208] Determine the ratio of the number of 1s in each result bitmap to the number of second values ​​in the access time bitmap in the access table pattern array as the matching degree between the access table pattern array and each access path pattern array;

[0209] The access path pattern arrays whose matching degrees are within a preset matching threshold range are determined as the access path pattern array list associated with the access table pattern array list.

[0210] The preset matching threshold range can be a fixed ratio, for example, determining the access path pattern arrays with matching degrees in the top 20% as the access path pattern array list associated with the access table pattern array list; the preset matching threshold range can also be a fixed number, for example, determining the access path pattern arrays with matching degrees in the top 5 as the access array pattern array list associated with the access table pattern array list. The preset matching threshold range can be set by the user and is not limited here. Any feasible method for determining the preset matching threshold range is within the scope of protection requested by this application.

[0211] The bitwise AND operation involved in the above process means that the result of the bitwise AND operation is 1 only when the result of the access time bitmap in the access table pattern array and the result of the extended access time bitmap in the access path pattern array are both 1. That is, 1 & 1 == 1, 1 & 0 == 0, 0 & 1 == 0, and 0 & 0 == 1. In other words, each bit in the access time bitmap result in the access table pattern array is ANDed with each bit in the extended access time bitmap result in each access path pattern array to obtain the number of 1s in each result bitmap.

[0212] In a specific embodiment, for each access table pattern sextet (data processing action, access table identifier, access time bitmap, extended access time bitmap, access requester identifier list, access responder identifier list), several associated access path pattern sextets (access action, access path, access time bitmap, extended access time bitmap, access requester identifier list, access responder identifier list) are filtered out according to the matching degree between the access time bitmap and the extended access time bitmap.

[0213] The specific steps are as follows:

[0214] (1) Count the number of 1s in the access time bitmap of the access table pattern 6-tuple, denoted as P;

[0215] (2) Perform a bitwise AND operation on the access time bitmap in the access table pattern sextuple and the extended access time bitmap of each access path pattern sextuple (assuming the number of access path pattern sextuples is K), and count the number of 1s in the resulting bitmap, which is recorded as Q1, Q2, ..., Q K ;

[0216] (3) Q1, Q2, ..., Q K Divide by P respectively to get the matching degree V1, V2, ..., V of the 1st to Kth access path pattern six-tuples. K , where V1=Q1 / P、V2=Q2 / P、……V K =Q K / P;

[0217] (4) Select V1, V2, ..., V K The corresponding access path pattern six-tuples of the plurality of largest median values ​​are recorded as associated access path pattern six-tuples of the access table pattern six-tuple, forming an associated access path pattern six-tuple list.

[0218] The purpose of using access time bitmap and extended access time bitmap is to efficiently calculate the frequency of "access paths and access tables between HTTP log data and database log data appearing in the same time period". This eliminates the need for frequent log loops required by traditional calculations, thereby improving computing performance.

[0219] Step S3 associates the access path pattern array list and the access table pattern array list through the time correlation between HTTP access events and database access events, and adopts a cross-comparison method to obtain the access table pattern array list associated with the access path pattern array and the access path pattern array list associated with the access table pattern array list. That is, relying on the correlation of access behavior patterns between different types of servers, the data flow association relationship between the HTTP server and the database server is automatically discovered, providing assistance for data space mapping and data security situation awareness.

[0220] In some specific implementations, after obtaining the access table pattern array list associated with the access path pattern array and the access path pattern array list associated with the access table pattern array list, each access path pattern array and the associated access table pattern array list can be stored, and at the same time, each access table pattern array and the associated access path pattern array can also be stored.

[0221] The data interaction path association reasoning method based on multi-source heterogeneous logs disclosed in this embodiment generates an HTTP interaction log array list based on the log data of the HTTP server, constructs a corresponding time bitmap according to the access time list in the HTTP interaction log array list, and generates an access path pattern array list based on the time bitmap and the HTTP interaction log array list; generates a database interaction log array list based on the log data of the database, constructs a corresponding time bitmap according to the access time list in the database interaction log array list, and generates an access table pattern array list based on the time bitmap and the database interaction log array list; performs association reasoning on the access path pattern array list and the access table pattern array list, and determines the association array list between the access path pattern list and the access table pattern list. The present invention obtains the flow path of data on the HTTP server and the database server by associating HTTP request analysis and database request analysis through time, thereby obtaining more comprehensive data information, improving the accuracy of data tracing, and further strengthening the ability of data security supervision.

[0222] Example 2:

[0223] Based on the same inventive concept, the present invention also provides a data interaction path association reasoning system 200 for multi-source heterogeneous logs. The system structure is as follows: Figure 2 As shown, the system 200 includes: an HTTP processing unit 201, a database processing unit 202, and an association reasoning unit 203; the HTTP processing unit is connected to the association reasoning unit, and the database processing unit is connected to the association reasoning unit;

[0224] HTTP processing unit 201 is used to parse the log data of the HTTP server, generate an HTTP interaction log array list, construct a corresponding time bitmap based on the access time list in the HTTP interaction log array list, generate an access path pattern array list based on the time bitmap and the HTTP interaction log array list, and send the access path pattern array list to the association reasoning unit;

[0225] The database processing unit 202 is configured to parse the database log data, generate a database interaction log array list, construct a corresponding time bitmap based on the access time list in the database interaction log array list, generate an access table pattern array list based on the time bitmap and the database interaction log list, and send the access table pattern array list to the association reasoning unit;

[0226] The association reasoning unit 203 is used to receive the access path pattern array list and the access table pattern array list, perform association reasoning on the access path pattern array list and the access table pattern array list, and determine the association array list between the access path pattern array list and the access table pattern array list.

[0227] Preferably, the HTTP processing unit 201 is specifically configured to:

[0228] Parse each log data of the HTTP server to obtain each parsed HTTP log data;

[0229] Normalize each HTTP log data after parsing to obtain each HTTP log array;

[0230] Get the HTTP interaction log array list based on each HTTP log array.

[0231] Preferably, the HTTP processing unit 201 is specifically configured to:

[0232] Each HTTP log data after parsing includes timestamp, source IP, source port, destination IP, destination port, request URL, request length, response code, and response length;

[0233] Each HTTP log array includes access time, access action, access path, access requester ID, and access responder ID;

[0234] Normalization of each parsed HTTP log data includes:

[0235] Generate access time based on the timestamp of HTTP log data; generate access action based on the legal request method of HTTP protocol in HTTP log data; generate access path based on the URL in HTTP request in HTTP log data, which access path includes URL path and request parameters; generate access requester identification based on the source IP of HTTP request in HTTP log data; generate access responder identification based on the destination IP and destination port of HTTP log data.

[0236] Preferably, the HTTP processing unit 201 is specifically configured to:

[0237] Modify the parameter value of the request parameter of the access path in the HTTP interaction log array list to the parameter name, and obtain the HTTP interaction log array list after parameter processing;

[0238] Aggregate the parameter-processed HTTP interaction log array list according to the access action and access path tuples, construct the corresponding access time list, access requester ID list, and access responder ID list for each access action and access path tuple, and generate each access path pattern set;

[0239] Constructing a corresponding time bitmap based on the access time list in each access path pattern set, and generating a corresponding access path pattern array; wherein the time bitmap includes an access time bitmap and an extended access time bitmap;

[0240] Generate an access path pattern array list according to each access path pattern array.

[0241] Preferably, the database processing unit 202 is specifically configured to:

[0242] Based on the database log data, generate a database interaction log array list, including:

[0243] Parse each log data in the database to obtain the parsed database log data;

[0244] Normalize each database log data after parsing to obtain each database log array;

[0245] Get the database interaction log array list based on each database log array.

[0246] Preferably, the database processing unit 202 is specifically configured to:

[0247] The parsed log data of each database includes timestamp, source IP, source port, destination IP, destination port, and request information;

[0248] Each database log array includes access time, data processing action, access table identifier, access requester identifier, and access responder identifier;

[0249] Normalization of the parsed log data of each database includes:

[0250] The access time is generated based on the timestamp of the database log data; the statements in the log are converted into an abstract syntax tree in a statement syntax analyzer to extract the data processing action; the statements in the log are converted into an abstract syntax tree in a statement syntax analyzer to extract the access table identifier; the access requester identifier is generated based on the source IP of the database request in the database log data; and the access responder identifier is generated based on the destination IP and destination port of the database log data.

[0251] Preferably, the database processing unit 202 is specifically configured to:

[0252] Aggregate the database interaction log array list according to the data processing action and access table identifier tuple, construct the corresponding access time list, access requester identifier list, and access responder identifier list for each data processing action and access table identifier tuple, and generate each access table pattern set;

[0253] Constructing a corresponding time bitmap based on the access time list in each access table pattern set, and generating a corresponding access table pattern array; wherein the time bitmap includes an access time bitmap and an extended access time bitmap;

[0254] Generate an access table pattern array list according to each access table pattern array.

[0255] Preferably, the association reasoning unit 203 is specifically used to:

[0256] Based on each access path pattern array, according to the matching degree between the time bitmap in the access path pattern array and the time bitmap in each access table pattern array, a list of access table pattern arrays associated with the access path pattern array is screened out.

[0257] Preferably, the association reasoning unit 203 is specifically used to:

[0258] Perform a bitwise AND operation on the access time bitmap in the access path pattern array and the extended access time bitmap in each access table pattern array to obtain the number of 1s in each resulting bitmap;

[0259] Determine the ratio of the number of 1s in each result bitmap to the number of second values ​​in the access time bitmap in the access path pattern array as the matching degree between the access path pattern array and each access table pattern array;

[0260] The access table pattern arrays whose matching degrees are within a preset matching threshold range are determined as the access table pattern array list associated with the access path pattern array list.

[0261] Preferably, the association reasoning unit 203 is specifically used to:

[0262] Based on each access table pattern array, according to the matching degree between the time bitmap in the access table pattern array and the time bitmap in each access path pattern array, an access path pattern array list associated with the access table pattern array is screened out.

[0263] Preferably, the association reasoning unit 203 is specifically used to:

[0264] Perform a bitwise AND operation on the access time bitmap in the access table pattern array and the extended access time bitmap in each access path pattern array to obtain the number of 1s in each resulting bitmap;

[0265] Determine the ratio of the number of 1s in each result bitmap to the number of second values ​​in the access time bitmap in the access table pattern array as the matching degree between the access table pattern array and each access path pattern array;

[0266] The access path pattern arrays whose matching degrees are within a preset matching threshold range are determined as the access path pattern array list associated with the access table pattern array list.

[0267] Preferably, the HTTP processing unit 201 is specifically configured to:

[0268] Construct a corresponding access time bitmap based on the access time list in each access path pattern set, including:

[0269] Select a time period that includes the access time in each access path pattern set and the access time in each access table pattern set, and number the time periods consecutively starting from 1 with one-second intervals to obtain a time period number;

[0270] Determine the size of the memory space according to the maximum number in the time period number, and set all initial values ​​in the memory space to the first value;

[0271] Traverse the access time in each access path pattern set, convert the access time of each access path pattern set into a time period number, set the time period number corresponding to the access time to the second value in the corresponding number position in the memory space, and obtain the access time bitmap corresponding to each access path pattern set.

[0272] Preferably, the HTTP processing unit 201 is specifically configured to:

[0273] Constructing a corresponding extended access time bitmap based on the access time list in each access path pattern set includes:

[0274] Predetermine the number of seconds to delay the associated database access event;

[0275] The number of rated digits following each second value of the access time bitmap is set to the second value, where the rated digits are the number of seconds corresponding to the delay in allowing the associated database access event;

[0276] An extended access time bitmap of the access path pattern array is determined according to the access time bitmap after being set to the second value.

[0277] Preferably, the database processing unit 202 is specifically configured to:

[0278] Constructing a corresponding access time bitmap based on the access time list in each access table pattern set includes:

[0279] Select a time period that includes the access time in each access table pattern set and the access time in each access path pattern set, and number the time periods consecutively starting from 1 with one-second intervals to obtain a time period number;

[0280] Determine the size of the memory space according to the maximum number in the time period number, and set all initial values ​​in the memory space to the first value;

[0281] Traverse the access time in each access table pattern set, convert the access time of each access table path pattern set into a time period number, set the time period number corresponding to the access time to the second value in the corresponding number position in the memory space, and obtain the access time bitmap corresponding to each access table pattern set.

[0282] Preferably, the database processing unit 202 is specifically configured to:

[0283] Constructing a corresponding extended access time bitmap based on the access time list in each access table pattern set includes:

[0284] Predetermine the number of seconds allowed for the associated HTTP request event to advance;

[0285] The first rated number of digits of each second value in the access time bitmap is set to the second value, where the rated number of digits corresponds to the number of seconds allowed for the associated HTTP request event to advance;

[0286] An extended access time bitmap of the access table pattern array is determined according to the access time bitmap after being set to the second value.

[0287] Example 3:

[0288] Based on the same inventive concept, Figure 3 As shown, the present invention also provides an electronic device, which may be a computer, a single-chip microcomputer, a smart mobile device, or the like. The electronic device in this embodiment may include a processor, a memory, a transceiver component, and the like. The memory, processor, and transceiver component are connected via a bus; the memory may be used to store an execution program, which may include instructions; and the processor may be used to execute the instructions stored in the memory. The memory may also be used to store data, which may be accessed and / or modified during the execution of the instructions.

[0289] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in a readable storage medium to implement the corresponding method flow or corresponding function, so as to implement the steps of a data interaction path association reasoning method based on multi-source heterogeneous logs in the above embodiment.

[0290] Example 4:

[0291] Based on the same inventive concept, the present invention also provides a readable storage medium, specifically an electronic device readable storage medium (Memory), which is a memory device in an electronic device for storing programs and data. It can be understood that the readable storage medium here can include both built-in storage media in the electronic device and, of course, extended storage media supported by the electronic device. The storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space. These instructions can be one or more execution programs (including program code). It should be noted that the storage medium here can be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk storage. The processor can load and execute one or more instructions stored in the storage medium to implement the steps of a data interaction path association reasoning method based on multi-source heterogeneous logs in the above embodiment.

[0292] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0293] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0294] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0295] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0296] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit its scope of protection. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that after reading the present invention, those skilled in the art may still make various changes, modifications or equivalent substitutions to the specific implementation methods of the application, but these changes, modifications or equivalent substitutions are all within the scope of protection of the pending claims.< / filename> < / filename> < / filename>

Claims

1. A data interaction path association reasoning method based on multi-source heterogeneous logs, characterized in that: The method comprises: Generate an HTTP interaction log array list based on the log data of the HTTP server, construct a corresponding time bitmap according to the access time list in the HTTP interaction log array list, and generate an access path pattern array list according to the time bitmap and the HTTP interaction log array list; Generate a database interaction log array list based on the log data of the database, construct a corresponding time bitmap according to the access time list in the database interaction log array list, and generate an access table pattern array list according to the time bitmap and the database interaction log array list; Associative reasoning is performed on the access path pattern array list and the access table pattern array list to determine an associative array list between the access path pattern list and the access table pattern list.

2. The method according to claim 1, characterized in that The HTTP interaction log array list is generated based on the log data of the HTTP server, including: Parse each piece of log data of the HTTP server to obtain each piece of HTTP log data after parsing; Normalize each HTTP log data after parsing to obtain each HTTP log array; The HTTP interaction log array list is obtained according to each HTTP log array.

3. The method according to claim 2, characterized in that Each piece of HTTP log data after parsing includes timestamp, source IP, source port, destination IP, destination port, request URL, request length, response code, and response length; Each HTTP log array includes access time, access action, access path, access requester ID, and access responder ID; The normalization processing of each HTTP log data after parsing includes: The access time is generated based on the timestamp of the HTTP log data; the access action is generated based on the legal request method of the HTTP protocol of the HTTP log data; the access path is generated based on the URL in the HTTP request of the HTTP log data, and the access path includes the URL path and request parameters; the access requester identifier is generated based on the source IP of the HTTP request of the HTTP log data; and the access responder identifier is generated based on the destination IP and destination port of the HTTP log data.

4. The method according to claim 3, characterized in that The constructing a corresponding time bitmap according to the access time list in the HTTP interaction log array list, and generating an access path pattern array list according to the time bitmap and the HTTP interaction log array list comprises: Modify the parameter value of the request parameter of the access path in the HTTP interaction log array list to the parameter name, and obtain the HTTP interaction log array list after parameter processing; Aggregating the HTTP interaction log array list after parameter processing according to the access action and access path tuples, constructing a corresponding access time list, access requester identification list, and access responder identification list for each access action and access path tuple, and generating each access path pattern set; Constructing a corresponding time bitmap based on the access time list in each access path pattern set, and generating a corresponding access path pattern array; wherein the time bitmap includes an access time bitmap and an extended access time bitmap; The access path pattern array list is generated according to each access path pattern array.

5. The method according to claim 1, characterized in that The database interaction log array list is generated based on the log data of the database, including: Parse each log data of the database to obtain each database log data after parsing; Normalize each piece of database log data after parsing to obtain an array of each database log; The database interaction log array list is obtained according to each database log array.

6. The method according to claim 5, characterized in that The parsed log data of each database includes timestamp, source IP, source port, destination IP, destination port, and request information; Each database log array includes access time, data processing action, access table identifier, access requester identifier, and access responder identifier; The normalization process of the parsed log data of each database includes: The access time is generated based on the timestamp of the database log data; the statements in the log are converted into an abstract syntax tree in a statement syntax analyzer to extract the data processing action; the statements in the log are converted into an abstract syntax tree in a statement syntax analyzer to extract the access table identifier; the access requester identifier is generated based on the source IP of the database request of the database log data; the access responder identifier is generated based on the destination IP and destination port of the database log data.

7. The method according to claim 6, characterized in that The constructing a corresponding time bitmap according to the access time list in the database interaction log array list, and generating an access table pattern array list according to the time bitmap and the database interaction log array list comprises: Aggregate the database interaction log array list according to the data processing action and access table identifier tuple, construct a corresponding access time list, access requester identifier list, and access responder identifier list for each data processing action and access table identifier tuple, and generate each access table pattern set; Constructing a corresponding time bitmap based on the access time list in each access table pattern set, and generating a corresponding access table pattern array; wherein the time bitmap includes an access time bitmap and an extended access time bitmap; The access table pattern array list is generated according to each access table pattern array.

8. The method according to claim 4, characterized in that The performing associative reasoning on the access path pattern array list and the access table pattern array list to determine an associative array list between the access path pattern list and the access table pattern list comprises: Based on each of the access path pattern arrays, according to the matching degree between the time bitmap in the access path pattern array and the time bitmap in each of the access table pattern arrays, a list of access table pattern arrays associated with the access path pattern array is screened out.

9. The method according to claim 8, characterized in that The step of screening out an access table pattern array list associated with the access path pattern array according to the matching degree between the time bitmap in the access path pattern array and the time bitmap in each access table pattern array comprises: Performing a bitwise AND operation on the access time bitmap in the access path pattern array and the extended access time bitmap in each of the access table pattern arrays to obtain the number of 1s in each result bitmap; Determine the ratio of the number of 1s in each result bitmap to the number of second values ​​in the access time bitmap in the access path pattern array as the matching degree between the access path pattern array and each access table pattern array; The access table pattern arrays whose matching degree is within a preset matching threshold range are determined as the access table pattern array list associated with the access path pattern array list.

10. The method according to claim 7, characterized in that The performing associative reasoning on the access path pattern array list and the access table pattern array list to determine an associative array list between the access path pattern list and the access table pattern list comprises: Based on each of the access table pattern arrays, an access path pattern array list associated with the access table pattern array is screened out according to the matching degree between the time bitmap in the access table pattern array and the time bitmap in each of the access path pattern arrays.

11. The method according to claim 10, characterized in that The step of screening out an access path pattern array list associated with the access table pattern array according to the matching degree between the time bitmap in the access table pattern array and the time bitmap in each access path pattern array includes: Performing a bitwise AND operation on the access time bitmap in the access table pattern array and the extended access time bitmap in each of the access path pattern arrays to obtain the number of 1s in each result bitmap; Determine the ratio of the number of 1s in each result bitmap to the number of second values ​​in the access time bitmap in the access table pattern array as the matching degree between the access table pattern array and each access path pattern array; The access path pattern arrays whose matching degree is within a preset matching threshold range are determined as the access path pattern array list associated with the access table pattern array list.

12. The method according to claim 4, characterized in that The constructing a corresponding access time bitmap based on the access time list in each access path pattern set includes: Select a time period including the access time in each of the access path pattern sets and the access time in each of the access table pattern sets, and number the time periods consecutively starting from 1 at intervals of one second to obtain a time period number; Determine the size of the memory space according to the maximum number in the time period numbers, and set all initial values ​​in the memory space to the first value; Traverse the access time in each of the access path pattern sets, convert the access time of each of the access path pattern sets into a time period number, set the time period number corresponding to the access time to a second value at the corresponding number position in the memory space, and obtain an access time bitmap corresponding to each of the access path pattern sets.

13. The method according to claim 12, characterized in that The constructing of a corresponding extended access time bitmap based on the access time list in each access path pattern set comprises: Predetermine the number of seconds to allow for delay in accessing events to the associated database; The number of rated digits following each second value of the access time bitmap is set to the second value, wherein the rated digits is the number corresponding to the number of seconds that the associated database access event is allowed to lag behind; An extended access time bitmap of the access path pattern array is determined according to the access time bitmap after being set to a second value.

14. The method according to claim 7, characterized in that The constructing of a corresponding access time bitmap based on the access time list in each access table pattern set comprises: Select a time period including the access time in each of the access table pattern sets and the access time in each of the access path pattern sets, and number the time periods consecutively starting from 1 at intervals of one second to obtain a time period number; Determine the size of the memory space according to the maximum number in the time period numbers, and set all initial values ​​in the memory space to the first value; Traverse the access time in each of the access table pattern sets, convert the access time of each of the access table path pattern sets into a time period number, set the time period number corresponding to the access time to a second value at the corresponding number position in the memory space, and obtain an access time bitmap corresponding to each of the access table pattern sets.

15. The method according to claim 14, characterized in that The constructing of a corresponding extended access time bitmap based on the access time list in each access table pattern set comprises: Predetermine the number of seconds allowed for the associated HTTP request event to be advanced; The first rated number of digits of each second value in the access time bitmap is set to the second value, wherein the rated number of digits is the number corresponding to the number of seconds that the associated HTTP request event is allowed to advance; An extended access time bitmap of the access table pattern array is determined according to the access time bitmap after being set to a second value.

16. A data interaction path association reasoning system based on multi-source heterogeneous logs, characterized in that: The system comprises: an HTTP processing unit, a database processing unit, and an association reasoning unit; the HTTP processing unit is connected to the association reasoning unit, and the database processing unit is connected to the association reasoning unit; An HTTP processing unit is used to parse the log data of the HTTP server, generate an HTTP interaction log array list, construct a corresponding time bitmap according to the access time list in the HTTP interaction log array list, generate an access path pattern array list according to the time bitmap and the HTTP interaction log array list, and send the access path pattern array list to the association reasoning unit; A database processing unit, used to parse the log data of the database, generate a database interaction log array list, construct a corresponding time bitmap according to the access time list in the database interaction log array list, generate an access table pattern array list according to the time bitmap and the database interaction log list, and send the access table pattern array list to the association reasoning unit; An associative reasoning unit is used to receive the access path pattern array list and the access table pattern array list, perform associative reasoning on the access path pattern array list and the access table pattern array list, and determine an associative array list between the access path pattern array list and the access table pattern array list.

17. The system according to claim 16, characterized in that The HTTP processing unit is used for: Parse each piece of log data of the HTTP server to obtain each piece of HTTP log data after parsing; Normalize each HTTP log data after parsing to obtain each HTTP log array; The HTTP interaction log array list is obtained according to each HTTP log array.

18. The system according to claim 16, characterized in that The database processing unit is used for: Parse each log data of the database to obtain each database log data after parsing; Normalize each piece of database log data after parsing to obtain an array of each database log; The database interaction log array list is obtained according to each database log array.

19. An electronic device, characterized in that: include: at least one processor and memory; The memory and the processor are connected via a bus; The memory is used to store one or more programs; When the one or more programs are executed by the at least one processor, the data interaction path association reasoning method based on multi-source heterogeneous logs as described in any one of claims 1 to 15 is implemented.

20. A readable storage medium, characterized in that: An execution program is stored thereon, and when the execution program is executed, the data interaction path association reasoning method based on multi-source heterogeneous logs as described in any one of claims 1 to 15 is implemented.