Data query statement processing method and device based on log data

Abstract

The invention discloses a data query statement processing method and device based on log data. The method comprises the steps that after operation detail data of a to-be-processed data query statement is obtained, all candidate service logs of the to-be-processed data query statement are obtained; determining the matching degree of each candidate business log; determining association confidence of each candidate business log; detecting whether an associated service log of the to-be-processed data query statement exists in each candidate service log or not; and if the associated business log exists, determining a business operation type associated with the to-be-processed data query statement according to the associated business log, and storing the to-be-processed data query statement and the associated business operation type. According to the embodiment of the invention, the business operation type associated with the data query statement can be quickly and accurately determined automatically based on the operation detail data and the business log, and the data query statement and the business operation type associated with the data query statement are stored.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method and apparatus for processing data query statements based on log data. Background Technology

[0002] Enterprises typically have multiple application servers. Each application server can be used to manage a specific business function within the enterprise. During the operation of an application server, it generates business logs. Business logs are text files generated by the application server describing a business processing operation performed by the server. The business logs contain the business operation type of the business processing operation. During the execution of a business processing operation by the application server, when the application server needs to retrieve specific business data from the database server, it inputs a data query statement to the database server to retrieve the specified business data. The database server runs the data query statement, retrieves the specified business data from the database, and provides it to the application server. After each data query statement is run by the database server, it needs to be processed promptly to determine the business operation type associated with the query statement. This allows operations personnel to determine whether the operation that triggered the database server to run the data query statement is a reasonable business processing operation. The business operation type associated with the data query statement is the business operation type that triggers the database server to run the data query statement.

[0003] In related technologies, a common data query statement processing scheme involves processing the data query statement based on business identification information transmitted synchronously with it to determine the business operation type associated with the query statement. However, this business identification information is easily lost during transmission. Therefore, the data query statement processing schemes in related technologies rely on this easily lost business identification information, making it impossible to effectively and reliably process the data query statement promptly after each execution on the database server to determine the associated business operation type. Summary of the Invention

[0004] This invention provides a data query statement processing method and apparatus based on log data to solve the problem that data query statement processing schemes in related technologies rely on easily lost business identification information, and cannot effectively and stably process data query statements in a timely manner after each execution of a data query statement on the database server, and determine the business operation type associated with the data query statement.

[0005] According to one aspect of the present invention, a method for processing data query statements based on log data is provided, comprising: After obtaining the operation details data of the data query statement to be processed, the candidate business logs of the data query statement to be processed are obtained from the business log storage component. Based on the operation details data and the data in each candidate business log, the matching degree of each candidate business log is determined; wherein, the matching degree includes time matching degree, resource matching degree, parameter matching degree and semantic matching degree; Based on the matching degree of each candidate business log, determine the association confidence degree of each candidate business log; Based on the association confidence of each candidate business log, detect whether there is an associated business log of the data query statement to be processed in each candidate business log; If there is an associated business log for the data query statement to be processed, then the business operation type associated with the data query statement to be processed is determined based on the associated business log, and the data query statement to be processed and the business operation type associated with the data query statement to be processed are stored.

[0006] According to another aspect of the present invention, a data query statement processing apparatus based on log data is provided, comprising: The log acquisition module is used to obtain each candidate business log of the data query statement to be processed from the business log storage component after obtaining the operation details data of the data query statement to be processed. The matching degree determination module is used to determine the matching degree of each candidate business log based on the operation details data and the data in each candidate business log; wherein, the matching degree includes time matching degree, resource matching degree, parameter matching degree and semantic matching degree; The confidence level determination module is used to determine the association confidence level of each candidate business log based on the matching degree of each candidate business log. The log detection module is used to detect whether there are related business logs of the data query statement to be processed in each candidate business log based on the correlation confidence of each candidate business log. The statement association module is used to determine the business operation type associated with the data query statement to be processed based on the associated business log if there is an associated business log for the data query statement to be processed, and to store the data query statement to be processed and the business operation type associated with the data query statement to be processed.

[0007] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that is executed by the at least one processor, which enables the at least one processor to execute the data query statement processing method based on log data according to any embodiment of the present invention.

[0008] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the data query statement processing method based on log data as described in any embodiment of the present invention.

[0009] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the data query statement processing method based on log data as described in any embodiment of the present invention.

[0010] The technical solution of this invention involves obtaining the operation details data of the data query statement to be processed from the business log storage component after obtaining the operation details data; then, determining the matching degree of each candidate business log based on the operation details data and the data in each candidate business log; wherein the matching degree includes time matching degree, resource matching degree, parameter matching degree, and semantic matching degree; determining the association confidence degree of each candidate business log based on the matching degree of each candidate business log; detecting whether there are associated business logs of the data query statement to be processed in each candidate business log based on the association confidence degree of each candidate business log; if there are associated business logs of the data query statement to be processed, determining the business operation associated with the data query statement to be processed based on the associated business logs. This system stores the data query statement to be processed and the business operation type associated with it. It solves the problem in related technologies where data query processing schemes rely on easily lost business identifier information, making it impossible to effectively and reliably process data queries and determine the associated business operation type after each execution of a data query on the database server. It can automatically and quickly determine the associated business operation type based on operation details and business logs, storing both the data query statement and its associated business operation type. This enables effective and stable processing of data queries and timely determination of their associated business operation types after execution on the database server.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart of a data query statement processing method based on log data, provided in Embodiment 1 of the present invention.

[0014] Figure 2 This is a flowchart of a data query statement processing method based on log data, provided in Embodiment 2 of the present invention.

[0015] Figure 3 This is a schematic diagram of a data query statement processing device based on log data provided in Embodiment 3 of the present invention.

[0016] Figure 4 A schematic diagram of the structure of an electronic device for implementing the data query statement processing method based on log data according to an embodiment of the present invention. Detailed Implementation

[0017] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0018] It should be noted that the terms "target," "first," "second," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising," "including," and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0019] Example 1 Figure 1 This is a flowchart illustrating a data query statement processing method based on log data, provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where, after a data query statement is executed on a database server, the type of business operation associated with the data query statement is determined. This method can be executed by a log data-based data query statement processing device, which can be implemented in hardware and / or software and can be configured in an electronic device installed within an enterprise. The electronic device can be an electronic device installed within the enterprise used to determine the type of business operation associated with the data query statement after it has been executed on the database server. For example... Figure 1 As shown, the method includes: Step 101: After obtaining the operation details data of the data query statement to be processed, retrieve each candidate business log of the data query statement to be processed from the business log storage component.

[0020] Optionally, an enterprise typically has multiple application servers. Each application server can be a server used to manage a specific business function of the enterprise. During the operation of the application servers, business logs are generated. Each business log can be text generated by the application server describing a business processing operation performed by the application server. A business processing operation can be an operation performed by the application server related to a specific business function of the enterprise managed by the application server. The specific business function of the enterprise managed by the application server can be producing software products with specific functions, producing hardware products with specific functions, providing specific services to specific individuals, or providing specific services to specific other enterprises. The business logs contain the business operation type of the business processing operation. The business operation type can be text used to characterize the type of business processing operation described in the business log. Each application server connects to a database server set up within the enterprise and can interact with the database server. The database server is a server set up within the enterprise to store business data. The database server contains databases. Business data is stored in the databases set up within the database server. Business data can be data related to the enterprise's business. Data query statements can be code statements that, after execution, are used to retrieve specific business data from the databases set up within the database server. Database servers can retrieve specific business data from the database by running data query statements, and then provide the specified business data to the application servers that need to retrieve the specified business data.

[0021] Optionally, during the application server's execution of business processing operations, when the application server needs to retrieve specific business data from the database server, the application server will input a data query statement to the database server to retrieve the specific business data. The database server can then retrieve the specific business data from the database by running the data query statement and provide the specified business data to the application server.

[0022] Optionally, the business operation type associated with the data query statement can refer to the business operation type that triggers the database server to run the data query statement. After each data query statement is run by the database server, the business operation type associated with the data query statement needs to be determined promptly. This allows operations and maintenance personnel to determine whether the operation that triggered the database server to run the data query statement is a reasonable business processing operation.

[0023] Optionally, the data query statement to be processed can be a data query statement that requires determining the associated business operation type. The operation details data of the data query statement to be processed can be information generated by the database server to record the data query statement to be processed. The operation details data of the data query statement to be processed includes the data query statement to be processed, the execution start timestamp, the execution unit identifier, and the interaction identifier. The execution start timestamp in the operation details data can be the time recorded by the database server when the data query statement to be processed started running. The execution unit identifier in the operation details data can be the thread identifier and process identifier related to the business data queried by the data query statement to be processed. The interaction identifier in the operation details data can be the user identifier and session identifier related to the business data queried by the data query statement to be processed. The business data queried by the data query statement to be processed refers to the specified business data obtained from the database by running the data query statement. The thread identifier can be text used to identify a thread. The process identifier can be text used to identify a process. The user identifier can be text used to identify a user. The session identifier can be text used to identify the interaction process between the enterprise and the user.

[0024] Optionally, after each execution of a data query statement, the database server can identify the completed data query statement as a pending data query statement and send the operation details of the pending data query statement to an electronic device used to determine the business operation type associated with the data query statement after the database server executes the data query statement. It can detect whether the electronic device has received the operation details of the pending data query statement. After detecting that the electronic device has received the operation details of the pending data query statement, it can obtain the received operation details of the pending data query statement and then retrieve the various candidate business logs of the pending data query statement from the business log storage component.

[0025] Optionally, the business log storage component can be a hardware or software module installed in an electronic device for storing business logs generated by various application servers. Business logs generated by various application servers can be acquired and stored in the business log storage component. Each candidate business log for the pending data query statement can be multiple business logs in the business log storage component that may describe the business processing operation that triggered the database server to run the pending data query statement. The business logs contain business timestamps. The business timestamp can be the time recorded by the application server when the business processing operation described in the business log began execution. The preset time difference threshold can be a pre-set time difference threshold. Typically, business logs whose absolute value of the time difference between the included business timestamp and the execution start timestamp in the operation details data of the pending data query statement is less than or equal to the preset time difference threshold are likely business logs that describe the business processing operation that triggered the database server to run the pending data query statement. For example, the preset time difference threshold ranges from 100 milliseconds to 500 milliseconds. The business log storage component contains multiple business logs where the absolute value of the time difference between the business timestamp and the execution start timestamp in the operation details data of the pending data query statement is less than or equal to a preset time difference threshold.

[0026] Optionally, obtaining each candidate business log of the data query statement to be processed from the business log storage component includes: detecting whether each business log in the business log storage component is a candidate business log of the data query statement to be processed based on the execution start timestamp in the operation details data and the business timestamp in each business log in the business log storage component, and obtaining each detected candidate business log of the data query statement to be processed.

[0027] Optionally, based on the execution start timestamp in the operation details data and the business timestamps in each business log in the business log storage component, it is determined whether each business log in the business log storage component is a candidate business log for the data query statement to be processed. This includes: performing the following operation for each business log in the business log storage component: calculating the absolute value of the time difference between the business timestamp in the business log and the execution start timestamp in the operation details data of the data query statement to be processed; comparing the calculated absolute value of the time difference with a preset time difference threshold; if the calculated absolute value of the time difference is less than or equal to the preset time difference threshold, then the business log is determined to be a candidate business log for the data query statement to be processed; if the calculated absolute value of the time difference is greater than the preset time difference threshold, then the business log is determined not to be a candidate business log for the data query statement to be processed.

[0028] Step 102: Determine the matching degree of each candidate business log based on the operation details data and the data in each candidate business log.

[0029] The matching degree includes time matching degree, resource matching degree, parameter matching degree, and semantic matching degree.

[0030] Optionally, for each candidate business log, the matching degree can be a numerical value determined based on the operation details data of the data query statement to be processed and the data in the candidate business logs. This value measures the likelihood that the candidate business log is a related business log of the data query statement to be processed. The related business logs of the data query statement to be processed are business logs that describe the business processing operations that triggered the database server to run the data query statement to be processed. The matching degree includes time matching degree, resource matching degree, parameter matching degree, and semantic matching degree.

[0031] Optionally, the time matching degree of the candidate business log can be a numerical value determined based on the execution start timestamp in the operation details data and the business timestamp in the candidate business log. This value measures the likelihood that the candidate business log is a related business log of the data query statement to be processed. A higher time matching degree indicates a higher likelihood that the candidate business log is a related business log of the data query statement to be processed. A lower time matching degree indicates a lower likelihood that the candidate business log is a related business log of the data query statement to be processed.

[0032] Optionally, the business log includes an execution unit identifier and an interaction identifier. The execution unit identifier in the business log can be a thread identifier and a process identifier related to the business processing operation described in the business log. The interaction identifier in the business log can be a user identifier and a session identifier related to the business processing operation described in the business log. The resource matching degree of a candidate business log can be a numerical value determined based on the operation details data and the execution unit identifier and interaction identifier in the candidate business log, used to measure the probability that the candidate business log is a related business log of the data query statement to be processed. The higher the resource matching degree of a candidate business log, the higher the probability that the candidate business log is a related business log of the data query statement to be processed. The lower the resource matching degree of a candidate business log, the lower the probability that the candidate business log is a related business log of the data query statement to be processed.

[0033] Optionally, the key business parameters in the operation details data can refer to parameters included in the operation details data that describe the functions implemented by the business. Similarly, the key business parameters in the candidate business logs can refer to parameters included in the candidate business logs that describe the functions implemented by the business. The parameter matching degree of the candidate business logs can be a numerical value determined based on the key business parameters in the operation details data and the candidate business logs, used to measure the probability that the candidate business log is a related business log of the data query statement to be processed. A higher parameter matching degree indicates a higher probability that the candidate business log is a related business log of the data query statement to be processed. A lower parameter matching degree indicates a lower probability that the candidate business log is a related business log of the data query statement to be processed.

[0034] Optionally, the data query statement includes semantic key information. Semantic key information can be text that summarizes the function of the data query statement. Multiple preset statement templates are stored in the electronic device. Each preset statement template contains multiple different semantic key information that may be included in the data query statement. The expected statement template for the candidate business log can be the semantic key information that may be included in the data query statement triggered by the candidate business log selected from the various preset statement templates. The actual statement template corresponding to the operation detail data can be the semantic key information contained in the data query statement to be processed in the operation detail data. The semantic matching degree of the candidate business log can be a numerical value determined based on the expected statement template of the candidate business log and the actual statement template corresponding to the operation detail data, used to measure the probability that the candidate business log is a related business log of the data query statement to be processed. The higher the semantic matching degree of the candidate business log, the higher the probability that the candidate business log is a related business log of the data query statement to be processed. The lower the semantic matching degree of the candidate business log, the lower the probability that the candidate business log is a related business log of the data query statement to be processed.

[0035] Optionally, the matching degree of each candidate business log is determined based on the operation details data and the data in each candidate business log, including: determining the time matching degree of each candidate business log based on the execution start timestamp in the operation details data and the business timestamp in each candidate business log; determining the resource matching degree of each candidate business log based on the execution unit identifier and interaction identifier in the operation details data and each candidate business log; determining the parameter matching degree of each candidate business log based on the key business parameters in the operation details data and each candidate business log; and determining the semantic matching degree of each candidate business log based on the expected statement template of each candidate business log and the actual statement template corresponding to the operation details data.

[0036] Optionally, based on the execution start timestamp in the operation details data and the business timestamps in each candidate business log, the time matching degree of each candidate business log is determined, including: performing the following operation for each candidate business log: calculating the absolute value of the time difference between the business timestamp in the candidate business log and the execution start timestamp in the operation details data of the data query statement to be processed, and determining the absolute value of the calculated time difference as the time matching degree of the candidate business log.

[0037] Optionally, based on the operation details data and the execution unit identifier and interaction identifier in each candidate business log, the resource matching degree of each candidate business log is determined, including: performing the following operations for each candidate business log: detecting whether the operation details data of the candidate business log and the data query statement to be processed contain the same thread identifier, process identifier, user identifier, or session identifier; if the operation details data of the candidate business log and the data query statement to be processed contain the same thread identifier, process identifier, user identifier, or session identifier, then the resource matching degree of the selected business log is determined to be the target value; if the operation details data of the candidate business log and the data query statement to be processed do not contain the same thread identifier, process identifier, user identifier, or session identifier, then the resource matching degree of the selected business log is determined to be 0; wherein, the target value is a preset value, and the target value is greater than 0.

[0038] Optionally, the method further includes: detecting whether the candidate business logs contain identification information for restricted access data; if the candidate business logs contain identification information for restricted access data, it is determined that the data query statement to be processed cannot be associated with a business operation type, and the operation details data of the data query statement to be processed and each candidate business log of the data query statement to be processed are identified as security risk information, and the security risk information is provided to the target user; if the candidate business logs do not contain identification information for restricted access data, the operation of determining the parameter matching degree of each candidate business log based on the operation details data and the key business parameters in each candidate business log continues. Restricted access data can be business data stored in the database that is prohibited from being accessed. The identification information for restricted access data can be pre-set information used to identify restricted access data. Security risk information can refer to information related to operations that affect the data security of the database server. The target user can be an operations and maintenance personnel in the enterprise. The security risk information can be sent to the target user's terminal device so that the target user can locate the operations that affect the data security of the database server based on the security risk information.

[0039] Optionally, the electronic device includes a first parameter detection component. This first parameter detection component can be a software module used to analyze and detect detailed operation data to determine key business parameters within it. The input to the first parameter detection component is the detailed operation data. The output of the first parameter detection component is the key business parameters within the detailed operation data. Detailed operation data can be input into the first parameter detection component. The first parameter detection component analyzes and detects the detailed operation data, determines the key business parameters within it, and then outputs these key business parameters. The key business parameters output by the first parameter detection component can be obtained.

[0040] Optionally, the electronic device includes a second parameter detection component. This component can be a software module used to analyze and detect business logs to determine key business parameters within them. The input to the second parameter detection component is the business log. The output of the second parameter detection component is the key business parameters from the business log. Business logs can be input into the second parameter detection component. The component will analyze and detect the business logs, determine the key business parameters, and then output these parameters. The key business parameters output from the business log by the second parameter detection component can be obtained.

[0041] Optionally, the parameter matching degree of each candidate business log is determined based on the operation details data and the key business parameters in each candidate business log, including: inputting the operation details data of the data query statement to be processed into the first parameter detection component to obtain the key business parameters in the operation details data of the data query statement to be processed output by the first parameter detection component; inputting each candidate business log into the second parameter detection component to obtain the key business parameters in each candidate business log output by the second parameter detection component; and for each candidate business log, determining the parameter matching degree of the candidate business log based on the key business parameters in the candidate business log and the key business parameters in the operation details data of the data query statement to be processed.

[0042] Optionally, based on the key business parameters in the candidate business logs and the key business parameters in the operation details data of the data query statement to be processed, the parameter matching degree of the candidate business logs is determined, including: using a first similarity algorithm to calculate the similarity between the key business parameters in the candidate business logs and the key business parameters in the operation details data of the data query statement to be processed, and determining the calculated similarity as the parameter matching degree of the candidate business logs. The first similarity algorithm can be a pre-set algorithm used to calculate the similarity between the key business parameters in the business logs and the key business parameters in the operation details data of the data query statement.

[0043] Optionally, the parameter matching degree of the candidate business log is determined based on the key business parameters in the candidate business log and the key business parameters in the operation details data of the data query statement to be processed. This includes: detecting whether the key business parameters in the candidate business log and the key business parameters in the operation details data contain the same characters, recording each character that is contained in both the key business parameters in the candidate business log and the key business parameters in the operation details data; and determining the total number of recorded characters as the parameter matching degree of the candidate business log.

[0044] Optionally, the electronic device includes a template detection component. This component can be a software module used to analyze and detect candidate business logs to determine their expected statement templates. The input to the template detection component is the candidate business log. The output of the template detection component is the expected statement template of the candidate business log. Candidate business logs can be input into the template detection component. The component analyzes and detects the candidate business logs, determines their expected statement templates, and then outputs the expected statement templates. The expected statement templates of the candidate business logs output by the template detection component can be obtained.

[0045] Optionally, the electronic device includes a statement template component. This component can be a software module used to extract the actual statement template corresponding to the operation detail data from the operation detail data. The input to the statement template component is the operation detail data. The output of the statement template component is the actual statement template corresponding to the operation detail data. Operation detail data can be input into the statement template component. The statement template component will extract the actual statement template corresponding to the operation detail data from the operation detail data and then output the actual statement template corresponding to the operation detail data. The actual statement template corresponding to the operation detail data output by the statement template component can be obtained.

[0046] Optionally, the semantic matching degree of each candidate business log is determined based on the expected statement template of each candidate business log and the actual statement template corresponding to the operation detail data. This includes: inputting each candidate business log into a template detection component to obtain the expected statement template of each candidate business log output by the template detection component; inputting the operation detail data of the data query statement to be processed into a statement templateization component to obtain the actual statement template corresponding to the operation detail data output by the statement templateization component; for each candidate business log, using a second similarity algorithm to calculate the similarity between the expected statement template of the candidate business log and the actual statement template corresponding to the operation detail data, and determining the calculated similarity as the semantic matching degree of the candidate business log. The second similarity algorithm can be a pre-set algorithm for calculating the similarity between the expected statement template of the candidate business log and the actual statement template corresponding to the operation detail data.

[0047] Step 103: Determine the association confidence of each candidate business log based on the matching degree of each candidate business log.

[0048] Optionally, the association confidence of the candidate business log can be a value calculated based on the time matching degree, resource matching degree, parameter matching degree, and semantic matching degree of the candidate business log, which is used to comprehensively measure the probability that the candidate business log is an associated business log of the data query statement to be processed.

[0049] Optionally, the association confidence level of each candidate business log is determined based on the matching degree of each candidate business log. This includes determining the association confidence level of each candidate business log based on the confidence level calculation formula and the matching degree of each candidate business log. The confidence level calculation formula can be a pre-set formula used to calculate the association confidence level of candidate business logs.

[0050] Optionally, for each candidate business log, the following confidence calculation formula is used to determine the association confidence of the candidate business log based on its matching degree: Where Score represents the association confidence level of the candidate business log, and F... time It is the time matching degree of the candidate business logs, F resource It is the resource matching degree of the candidate business logs, F papram It is the parameter matching degree of the candidate business logs, F semantic This refers to the semantic matching degree of the candidate business logs. w1, w2, w3, and w4 are four preset coefficients. For example, w1 is 0.2, w2 is 0.3, w3 is 0.4, and w4 is 0.1.

[0051] Step 104: Based on the association confidence of each candidate business log, detect whether there is an associated business log of the data query statement to be processed in each candidate business log.

[0052] Optionally, the high confidence threshold and the low confidence threshold are two pre-set values. The high confidence threshold is greater than the low confidence threshold. Typically, candidate business logs with a correlation confidence greater than the high confidence threshold are the associated business logs of the data query statement to be processed. If the correlation confidence of all candidate business logs is less than the low confidence threshold, it can be determined that there are no associated business logs for the data query statement to be processed among the candidate business logs, thus confirming that the data query statement to be processed cannot be associated with a business operation type. "The data query statement to be processed cannot be associated with a business operation type" means that the business operation type associated with the data query statement to be processed cannot be determined. For example, the high confidence threshold ranges from 85 to 95, and the low confidence threshold ranges from 20 to 30.

[0053] Optionally, based on the association confidence of each candidate business log, detect whether there is an associated business log of the data query statement to be processed in each candidate business log, including: comparing the association confidence of each candidate business log with a high confidence threshold and a low confidence threshold; if there is a target candidate business log with an association confidence greater than the high confidence threshold, then determine that there is an associated business log of the data query statement to be processed in each candidate business log, and determine that the target candidate business log is an associated business log of the data query statement to be processed; if the association confidence of each candidate business log is less than the low confidence threshold... If the associated business logs of the pending data query statement are not found in any of the candidate business logs, it is determined that the pending data query statement cannot be associated with a business operation type. If the association confidence of each candidate business log is between the high confidence threshold and the low confidence threshold, the operation details data and each candidate business log are provided to the target user. After obtaining the recommended candidate business logs from the target user, it is determined that the associated business logs of the pending data query statement exist in each candidate business log, and the recommended candidate business logs are determined to be the associated business logs of the pending data query statement.

[0054] Optionally, the association confidence of each candidate business log can be compared with the high confidence threshold and the low confidence threshold to determine the relationship between the association confidence of each candidate business log and the high confidence threshold and the low confidence threshold.

[0055] Optionally, the target candidate business log is a candidate business log with a correlation confidence level greater than the high confidence threshold. If there are target candidate business logs with a correlation confidence level greater than the high confidence threshold, it can be determined that there are related business logs of the data query statement to be processed in each candidate business log, and the target candidate business log is determined to be the related business log of the data query statement to be processed.

[0056] Optionally, if the association confidence of each candidate business log is less than the low confidence threshold, it can be determined that there is no associated business log of the data query statement to be processed in each candidate business log, and that the data query statement to be processed cannot be associated with the business operation type, that is, it can be determined that the business operation type associated with the data query statement to be processed cannot be determined.

[0057] Optionally, recommended candidate business logs can refer to the associated business logs of the data query statements to be processed, selected by the target user from the candidate business logs after analyzing the operation details data and each candidate business log. If the association confidence of each candidate business log is between the high confidence threshold and the low confidence threshold, the operation details data and each candidate business log can be sent to the target user's terminal device. After receiving the operation details data and each candidate business log, the target user will analyze the operation details data and each candidate business log, determine the recommended candidate business logs, and then feed the recommended candidate business logs back to the electronic device through the terminal device. It is possible to obtain the recommended candidate business logs fed back by the target user, and then determine that there are associated business logs of the data query statements to be processed in each candidate business log, thus determining that the recommended candidate business logs are associated business logs of the data query statements to be processed. The association confidence of the candidate business logs being between the high confidence threshold and the low confidence threshold can mean that the association confidence of the candidate business logs is less than or equal to the high confidence threshold and the association confidence of the candidate business logs is greater than or equal to the low confidence threshold.

[0058] Step 105: If there is an associated business log for the data query statement to be processed, then determine the business operation type associated with the data query statement to be processed based on the associated business log, and store the data query statement to be processed and the business operation type associated with the data query statement to be processed.

[0059] Optionally, based on the associated business log, the business operation type associated with the data query statement to be processed is determined, including: extracting the business operation type from the associated business log of the data query statement to be processed, and determining the extracted business operation type as the business operation type associated with the data query statement to be processed.

[0060] Optionally, storing the data query statement to be processed and the business operation type associated with the data query statement includes: storing the data query statement to be processed and the business operation type associated with the data query statement in a statement mapping table. The statement mapping table can be a pre-set data table used to store data query statements and the business operation types associated with the data query statements. The target user can browse the data query statements and the business operation types associated with the data query statements in the statement mapping table, and determine whether the operation that triggers the database server to run the data query statement is a reasonable business processing operation based on the business operation type associated with the data query statement.

[0061] Optionally, the method further includes: if no associated business log exists for the data query statement to be processed, identifying the operation details data and associated process data as security risk information, and providing the security risk information to the target user. The associated process data may refer to each candidate business log of the data query statement to be processed, the matching degree of each candidate business log, and the association confidence degree of each candidate business log.

[0062] Optionally, the target user can analyze the stored data query statements and associated business operation types in the statement mapping table, as well as the received information related to the data query statements, to optimize the coefficients in the confidence calculation formula. The coefficients in the confidence calculation formula can be optimized and updated once after the electronic device processes a specified number of data query statements. For example, the specified number is 1000.

[0063] The technical solution of this invention involves obtaining the operation details data of the data query statement to be processed from the business log storage component after obtaining the operation details data; then, determining the matching degree of each candidate business log based on the operation details data and the data in each candidate business log; wherein the matching degree includes time matching degree, resource matching degree, parameter matching degree, and semantic matching degree; determining the association confidence degree of each candidate business log based on the matching degree of each candidate business log; detecting whether there are associated business logs of the data query statement to be processed in each candidate business log based on the association confidence degree of each candidate business log; if there are associated business logs of the data query statement to be processed, determining the business operation associated with the data query statement to be processed based on the associated business logs. This system stores the data query statement to be processed and the business operation type associated with it. It solves the problem in related technologies where data query processing schemes rely on easily lost business identifier information, making it impossible to effectively and reliably process data queries and determine the associated business operation type after each execution of a data query on the database server. It can automatically and quickly determine the associated business operation type based on operation details and business logs, storing both the data query statement and its associated business operation type. This enables effective and stable processing of data queries and timely determination of their associated business operation types after execution on the database server.

[0064] The technical solutions of this invention do not require large-scale modifications to existing application code, making them particularly suitable for legacy systems and third-party system integration scenarios, with low implementation costs. The technical solutions of this invention can integrate multi-dimensional evidence, maintaining a high correlation success rate even when data in some dimensions is missing. The technical solutions of this invention can automatically adapt to business changes and emerging calling patterns, reducing manual maintenance workload. The technical solutions of this invention can not only be used for security detection but also extended to multiple operational scenarios such as performance diagnostics, business auditing, and capacity planning. For example, they can be used to identify data query statement injection.

[0065] Example 2 Figure 2 This is a flowchart illustrating a data query statement processing method based on log data, provided in Embodiment 2 of the present invention. This embodiment of the present invention can be combined with various optional solutions from one or more of the above embodiments. For example... Figure 2 As shown, the method includes: Step 201: After obtaining the operation details data of the data query statement to be processed, retrieve each candidate business log of the data query statement to be processed from the business log storage component.

[0066] Step 202: Determine the time matching degree of each candidate business log based on the execution start timestamp in the operation details data and the business timestamp in each candidate business log.

[0067] Step 203: Determine the resource matching degree of each candidate business log based on the operation details data and the execution unit identifier and interaction identifier in each candidate business log.

[0068] Step 204: Determine the parameter matching degree of each candidate business log based on the operation details data and the key business parameters in each candidate business log.

[0069] Step 205: Determine the semantic matching degree of each candidate business log based on the expected statement template of each candidate business log and the actual statement template corresponding to the operation details data.

[0070] Step 206: Determine the association confidence of each candidate business log based on the time matching degree, resource matching degree, parameter matching degree, and semantic matching degree of each candidate business log.

[0071] Step 207: Based on the association confidence of each candidate business log, detect whether there is an associated business log of the data query statement to be processed in each candidate business log.

[0072] Step 208: If there is an associated business log for the data query statement to be processed, then determine the business operation type associated with the data query statement to be processed based on the associated business log, and store the data query statement to be processed and the business operation type associated with the data query statement to be processed.

[0073] The technical solution of this invention can automatically obtain each candidate business log of the data query statement from the business log storage component. Then, based on the operation details data and the data in each candidate business log, it determines the time matching degree, resource matching degree, parameter matching degree, and semantic matching degree of each candidate business log. Furthermore, based on the time matching degree, resource matching degree, parameter matching degree, and semantic matching degree of each candidate business log, it quickly and accurately determines the associated business log of the data query statement, determines the business operation type associated with the data query statement, and stores the data query statement and the business operation type associated with the data query statement. This enables effective and stable processing of the data query statement after it is executed on the database server, and timely determination of the business operation type associated with the data query statement.

[0074] Example 3 Figure 3 This is a schematic diagram of a data query statement processing device based on log data, provided in Embodiment 3 of the present invention. The device can be configured in an electronic device. Figure 3 As shown, the module includes a log acquisition module 301, a matching degree determination module 302, a confidence degree determination module 303, a log detection module 304, and a statement association module 305.

[0075] The system includes the following modules: a log acquisition module 301, which, after acquiring the operation details data of the data query statement to be processed, retrieves each candidate business log of the data query statement to be processed from the business log storage component; a matching degree determination module 302, which determines the matching degree of each candidate business log based on the operation details data and the data in each candidate business log; wherein the matching degree includes time matching degree, resource matching degree, parameter matching degree, and semantic matching degree; a confidence degree determination module 303, which determines the association confidence degree of each candidate business log based on the matching degree of each candidate business log; a log detection module 304, which detects whether there is an associated business log of the data query statement to be processed in each candidate business log based on the association confidence degree of each candidate business log; and a statement association module 305, which, if there is an associated business log of the data query statement to be processed, determines the business operation type associated with the data query statement to be processed based on the associated business log, and stores the data query statement to be processed and the business operation type associated with the data query statement to be processed.

[0076] The technical solution of this invention involves obtaining the operation details data of the data query statement to be processed from the business log storage component after obtaining the operation details data; then, determining the matching degree of each candidate business log based on the operation details data and the data in each candidate business log; wherein the matching degree includes time matching degree, resource matching degree, parameter matching degree, and semantic matching degree; determining the association confidence degree of each candidate business log based on the matching degree of each candidate business log; detecting whether there are associated business logs of the data query statement to be processed in each candidate business log based on the association confidence degree of each candidate business log; if there are associated business logs of the data query statement to be processed, determining the business operation associated with the data query statement to be processed based on the associated business logs. This system stores the data query statement to be processed and the business operation type associated with it. It solves the problem in related technologies where data query processing schemes rely on easily lost business identifier information, making it impossible to effectively and reliably process data queries and determine the associated business operation type after each execution of a data query on the database server. It can automatically and quickly determine the associated business operation type based on operation details and business logs, storing both the data query statement and its associated business operation type. This enables effective and stable processing of data queries and timely determination of their associated business operation types after execution on the database server.

[0077] In an optional embodiment of the present invention, the log acquisition module 301, when performing the operation of acquiring each candidate business log of the data query statement to be processed from the business log storage component, is specifically used to: detect whether each business log in the business log storage component is a candidate business log of the data query statement to be processed based on the execution start timestamp in the operation detail data and the business timestamp in each business log in the business log storage component, and acquire the detected candidate business logs of the data query statement to be processed.

[0078] In an optional embodiment of the present invention, the matching degree determination module 302 is specifically configured to: determine the time matching degree of each candidate business log based on the execution start timestamp in the operation detail data and the business timestamp in each candidate business log; determine the resource matching degree of each candidate business log based on the execution unit identifier and interaction identifier in the operation detail data and each candidate business log; determine the parameter matching degree of each candidate business log based on the key business parameters in the operation detail data and each candidate business log; and determine the semantic matching degree of each candidate business log based on the expected statement template of each candidate business log and the actual statement template corresponding to the operation detail data.

[0079] In an optional embodiment of the present invention, the confidence determination module 303 is specifically used to: determine the association confidence of each candidate business log based on the confidence calculation formula and the matching degree of each candidate business log.

[0080] In an optional embodiment of the present invention, the log detection module 304 is specifically configured to: compare the association confidence of each candidate business log with a high confidence threshold and a low confidence threshold; if there is a target candidate business log with an association confidence greater than the high confidence threshold, then it is determined that there is an associated business log of the data query statement to be processed in each candidate business log, and the target candidate business log is determined to be an associated business log of the data query statement to be processed; if the association confidence of each candidate business log is less than the low confidence threshold, then it is determined that there is no associated business log of the data query statement to be processed in each candidate business log, and the data query statement to be processed cannot be associated with a business operation type; if the association confidence of each candidate business log is between the high confidence threshold and the low confidence threshold, then the operation details data and each candidate business log are provided to the target user, and after obtaining the recommended candidate business logs fed back by the target user, it is determined that there is an associated business log of the data query statement to be processed in each candidate business log, and the recommended candidate business log is determined to be an associated business log of the data query statement to be processed.

[0081] In an optional embodiment of the present invention, the log detection module 304 is further configured to: if there is no associated business log for the data query statement to be processed, determine the operation details data and associated process data as security risk information, and provide the security risk information to the target user.

[0082] In an optional embodiment of the present invention, the statement association module 305 may, when performing the operation of storing the data query statement to be processed and the business operation type associated with the data query statement to be processed, specifically be used to: store the data query statement to be processed and the business operation type associated with the data query statement to be processed in a statement mapping table.

[0083] The log data-based data query statement processing device provided in this embodiment of the invention can execute the log data-based data query statement processing method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0084] Example 4 Figure 4 A schematic diagram of an electronic device 10, which can be used to implement the log data-based data query statement processing method of embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, electronic devices, blade electronic devices, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0085] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory 12 or a random access memory 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the read-only memory 12 or loaded from storage unit 18 into the random access memory 13. The random access memory 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, read-only memory 12, and random access memory 13 are interconnected via a bus 14. An input / output interface 15 is also connected to the bus 14.

[0086] Multiple components in electronic device 10 are connected to input / output interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of monitors, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0087] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, central processing units, graphics processing units, various special-purpose artificial intelligence computing chips, various processors running machine learning model algorithms, digital signal processors, and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as data query statement processing methods based on log data.

[0088] In some embodiments, the log data-based data query processing method can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as a storage unit. In some embodiments, part or all of the computer program can be loaded and / or installed on a heterogeneous hardware accelerator via read-only memory and / or a communication unit. When the computer program is loaded into random access memory and executed by a processor, one or more steps of the log data-based data query processing method described above can be performed. Alternatively, in other embodiments, the processor can be configured to execute the log data-based data query processing method by any other suitable means (e.g., by means of firmware).

[0089] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays, application-specific integrated circuits (ASICs), application-specific standard products (ASICs), systems-on-a-chip (SoCs), payload programmable logic devices (PLCs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0090] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or electronic device.

[0091] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, optical fibers, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0092] To provide user interaction, the systems and techniques described herein can be implemented on a heterogeneous hardware accelerator, which includes: a display device (e.g., a cathode ray tube or liquid crystal display monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the heterogeneous hardware accelerator. Other types of devices can also be used to provide user interaction; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or haptic feedback); and input from the user can be received in any form (including sound input, voice input, or haptic input).

[0093] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data electronic devices), or computing systems that include middleware components (e.g., application electronic devices), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0094] A computing system can include clients and electronic devices. Clients and electronic devices are generally geographically separated and typically interact via communication networks. The client-electronic device relationship is created by computer programs running on the respective computers and establishing a client-electronic device relationship between them. Electronic devices can be cloud electronic devices, also known as cloud computing electronic devices or cloud servers, which are host products within the cloud computing service system. These address the shortcomings of traditional physical hosts and virtual private server services, such as high management difficulty and weak business scalability.

[0095] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0096] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for processing data query statements based on log data, characterized in that, include: After obtaining the operation details data of the data query statement to be processed, the candidate business logs of the data query statement to be processed are obtained from the business log storage component. Based on the operation details data and the data in each candidate business log, the matching degree of each candidate business log is determined; wherein, the matching degree includes time matching degree, resource matching degree, parameter matching degree and semantic matching degree; Based on the matching degree of each candidate business log, determine the association confidence degree of each candidate business log; Based on the association confidence of each candidate business log, detect whether there is an associated business log of the data query statement to be processed in each candidate business log; If there is an associated business log for the data query statement to be processed, then the business operation type associated with the data query statement to be processed is determined based on the associated business log, and the data query statement to be processed and the business operation type associated with the data query statement to be processed are stored.

2. The data query statement processing method based on log data according to claim 1, characterized in that, Retrieve each candidate business log of the data query statement to be processed from the business log storage component, including: Based on the execution start timestamp in the operation details data and the business timestamps in each business log in the business log storage component, it is detected whether each business log in the business log storage component is a candidate business log for the data query statement to be processed, and the detected candidate business logs for the data query statement to be processed are obtained.

3. The data query statement processing method based on log data according to claim 1, characterized in that, Based on the operation details data and the data in each candidate business log, determine the matching degree of each candidate business log, including: Based on the execution start timestamp in the operation details data and the business timestamps in each candidate business log, determine the time matching degree of each candidate business log; Based on the operation details data and the execution unit identifier and interaction identifier in each candidate business log, determine the resource matching degree of each candidate business log; Based on the operation details data and the key business parameters in each candidate business log, determine the parameter matching degree of each candidate business log; The semantic matching degree of each candidate business log is determined based on the expected statement template of each candidate business log and the actual statement template corresponding to the operation details data.

4. The data query statement processing method based on log data according to claim 1, characterized in that, Based on the matching degree of each candidate business log, the association confidence of each candidate business log is determined, including: Based on the confidence calculation formula and the matching degree of each candidate business log, the association confidence of each candidate business log is determined.

5. The data query statement processing method based on log data according to claim 1, characterized in that, Based on the correlation confidence of each candidate business log, detect whether there is a related business log for the data query statement to be processed in each candidate business log, including: The correlation confidence of each candidate business log is compared with the high confidence threshold and the low confidence threshold. If there is a target candidate business log with a correlation confidence greater than the high confidence threshold, then it is determined that there is a related business log of the data query statement to be processed in each candidate business log, and the target candidate business log is determined to be the related business log of the data query statement to be processed. If the association confidence of each candidate business log is less than the low confidence threshold, it is determined that there is no associated business log of the data query statement to be processed in each candidate business log, and it is determined that the data query statement to be processed cannot be associated with the business operation type. If the association confidence of each candidate business log is between the high confidence threshold and the low confidence threshold, then the operation details data and each candidate business log are provided to the target user. After obtaining the recommended candidate business logs fed back by the target user, it is determined that there are related business logs of the data query statement to be processed in each candidate business log, and the recommended candidate business logs are determined to be the related business logs of the data query statement to be processed.

6. The data query statement processing method based on log data according to claim 5, characterized in that, Also includes: If there is no associated business log for the query statement to be processed, the operation details data and associated process data are identified as security risk information, and the security risk information is provided to the target user.

7. The data query statement processing method based on log data according to claim 1, characterized in that, The storage of the data query statement to be processed and the business operation type associated with the data query statement includes: The query statement to be processed and the business operation type associated with the query statement to be processed are stored in the statement mapping table.

8. A data query statement processing device based on log data, characterized in that, include: The log acquisition module is used to obtain each candidate business log of the data query statement to be processed from the business log storage component after obtaining the operation details data of the data query statement to be processed. The matching degree determination module is used to determine the matching degree of each candidate business log based on the operation details data and the data in each candidate business log; wherein, the matching degree includes time matching degree, resource matching degree, parameter matching degree and semantic matching degree; The confidence level determination module is used to determine the association confidence level of each candidate business log based on the matching degree of each candidate business log. The log detection module is used to detect whether there are related business logs of the data query statement to be processed in each candidate business log based on the correlation confidence of each candidate business log. The statement association module is used to determine the business operation type associated with the data query statement to be processed based on the associated business log if there is an associated business log for the data query statement to be processed, and to store the data query statement to be processed and the business operation type associated with the data query statement to be processed.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that is executed by the at least one processor, which enables the at least one processor to execute the data query statement processing method based on log data as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the data query statement processing method based on log data as described in any one of claims 1-7.

Images